diff --git a/intermediate_source/ddp_tutorial.rst b/intermediate_source/ddp_tutorial.rst index 680775123f8..1553cf1ac29 100644 --- a/intermediate_source/ddp_tutorial.rst +++ b/intermediate_source/ddp_tutorial.rst @@ -340,11 +340,12 @@ Let's still use the Toymodel example and create a file named ``elastic_ddp.py``. labels = torch.randn(20, 5).to(device_id) loss_fn(outputs, labels).backward() optimizer.step() + dist.destroy_process_group() if __name__ == "__main__": demo_basic() -One can then run a `torch elastic/torchrun`__ command +One can then run a `torch elastic/torchrun `__ command on all nodes to initialize the DDP job created above: .. code:: bash