From ea8f16c9f56696a4a217f59eaa256d1d7ff4599a Mon Sep 17 00:00:00 2001 From: Hugh Date: Wed, 2 Aug 2023 14:52:11 +0800 Subject: [PATCH] Update ddp_tutorial.rst 1. add `dist.destroy_process_group()` in example code block 2. modify the link syntax error about torchrun --- intermediate_source/ddp_tutorial.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/intermediate_source/ddp_tutorial.rst b/intermediate_source/ddp_tutorial.rst index 680775123f8..1553cf1ac29 100644 --- a/intermediate_source/ddp_tutorial.rst +++ b/intermediate_source/ddp_tutorial.rst @@ -340,11 +340,12 @@ Let's still use the Toymodel example and create a file named ``elastic_ddp.py``. labels = torch.randn(20, 5).to(device_id) loss_fn(outputs, labels).backward() optimizer.step() + dist.destroy_process_group() if __name__ == "__main__": demo_basic() -One can then run a `torch elastic/torchrun`__ command +One can then run a `torch elastic/torchrun `__ command on all nodes to initialize the DDP job created above: .. code:: bash