Skip to content

Commit ba97b64

Browse files
svekarssinhaanshul
authored andcommitted
Apply suggestions from code review
1 parent 4c6b4b1 commit ba97b64

File tree

1 file changed

+6
-4
lines changed

1 file changed

+6
-4
lines changed

recipes_source/distributed_comm_debug_mode.rst

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ Here is how you can use ``CommDebugMode``:
3434

3535
.. code-block:: python
3636
37+
# The model used in this example is a MLPModule applying Tensor Parallel
3738
comm_mode = CommDebugMode()
3839
with comm_mode:
3940
output = model(inp)
@@ -73,8 +74,8 @@ you want to use to display the data. You can also use a ``noise_level`` argument
7374
level of displayed information. Here is what each noise level displays:
7475

7576
| 0. Prints module-level collective counts
76-
| 1. Prints dTensor operations not included in trivial operations, module information
77-
| 2. Prints operations not included in trivial operations
77+
| 1. Prints DTensor operations (not including trivial operations), module sharding information
78+
| 2. Prints tensor operations (not including trivial operations)
7879
| 3. Prints all operations
7980
8081
In the example above, you can see that the collective operation, all_reduce, occurs once in the forward pass
@@ -196,8 +197,9 @@ Below is the interactive module tree visualization that you can use to upload yo
196197
Conclusion
197198
------------------------------------------
198199

199-
In this recipe, we have learned how to use ``CommDebugMode`` to debug Distributed Tensors. You can use your
200-
own JSON outputs in the embedded visual browser.
200+
In this recipe, we have learned how to use ``CommDebugMode`` to debug Distributed Tensors and
201+
parallelism solutions that uses communication collectives with PyTorch. You can use your own
202+
JSON outputs in the embedded visual browser.
201203

202204
For more detailed information about ``CommDebugMode``, see
203205
`comm_mode_features_example.py

0 commit comments

Comments
 (0)