We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent bf728cb commit b4e6207Copy full SHA for b4e6207
intermediate_source/ddp_tutorial.rst
@@ -340,11 +340,12 @@ Let's still use the Toymodel example and create a file named ``elastic_ddp.py``.
340
labels = torch.randn(20, 5).to(device_id)
341
loss_fn(outputs, labels).backward()
342
optimizer.step()
343
+ dist.destroy_process_group()
344
345
if __name__ == "__main__":
346
demo_basic()
347
-One can then run a `torch elastic/torchrun<https://pytorch.org/docs/stable/elastic/quickstart.html>`__ command
348
+One can then run a `torch elastic/torchrun <https://pytorch.org/docs/stable/elastic/quickstart.html>`__ command
349
on all nodes to initialize the DDP job created above:
350
351
.. code:: bash
0 commit comments