Skip to content

Commit cb37fe4

Browse files
authored
Apply suggestions from code review
1 parent 44c6e8d commit cb37fe4

File tree

1 file changed

+6
-7
lines changed

1 file changed

+6
-7
lines changed

recipes_source/distributed_comm_debug_mode.rst

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
1-
Using CommDebugMode
1+
Getting Started with ``CommDebugMode``
22
=====================================================
33

44
**Author**: `Anshul Sinha <https://github.com/sinhaanshul>`__
55

66

7-
In this tutorial, we will explore how to use CommDebugMode with PyTorch's
7+
In this tutorial, we will explore how to use ``CommDebugMode`` with PyTorch's
88
DistributedTensor (DTensor) for debugging by tracking collective operations in distributed training environments.
99

1010
Prerequisites
@@ -14,7 +14,7 @@ Prerequisites
1414
* PyTorch 2.2 or later
1515

1616

17-
What is CommDebugMode and why is it useful
17+
What is ``CommDebugMode`` and why is it useful
1818
------------------------------------------
1919
As the size of models continues to increase, users are seeking to leverage various combinations
2020
of parallel strategies to scale up distributed training. However, the lack of interoperability
@@ -32,7 +32,7 @@ users to view when and why collective operations are happening when using DTenso
3232
addressing this issue.
3333

3434

35-
How to use CommDebugMode
35+
Using ``CommDebugMode``
3636
------------------------
3737

3838
Here is how you can use ``CommDebugMode``:
@@ -56,10 +56,10 @@ Here is how you can use ``CommDebugMode``:
5656
# used in the visual browser below
5757
comm_mode.generate_json_dump(noise_level=2)
5858
59+
This is what the output looks like for a MLPModule at noise level 0:
60+
5961
.. code-block:: python
6062
61-
"""
62-
This is what the output looks like for a MLPModule at noise level 0
6363
Expected Output:
6464
Global
6565
FORWARD PASS
@@ -72,7 +72,6 @@ Here is how you can use ``CommDebugMode``:
7272
MLPModule.net2
7373
FORWARD PASS
7474
*c10d_functional.all_reduce: 1
75-
"""
7675
7776
To use ``CommDebugMode``, you must wrap the code running the model in ``CommDebugMode`` and call the API that
7877
you want to use to display the data. You can also use a ``noise_level`` argument to control the verbosity

0 commit comments

Comments
 (0)