Skip to content

Commit 6f9b767

Browse files
sinhaanshulsinhaanshul
committed
[dtensor][debug] tutorial showing users how to use commdebugmode and giving access to visual browser
1 parent c758b29 commit 6f9b767

File tree

1 file changed

+129
-9
lines changed

1 file changed

+129
-9
lines changed

‎distributed_comm_debug_mode.rst

Lines changed: 129 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -25,15 +25,135 @@ Using CommDebugMode and getting its output is very simple.
2525
with comm_mode:
2626
output = model(inp)
2727
28-
# print the operation level collective tracing information
29-
print(comm_mode.generate_comm_debug_tracing_table(noise_level=2))
28+
# print the operation level collective tracing information
29+
print(comm_mode.generate_comm_debug_tracing_table(noise_level=2))
3030
31-
# log the operation level collective tracing information to a file
32-
comm_mode.log_comm_debug_tracing_table_to_file(
33-
noise_level=1, file_name="transformer_operation_log.txt"
34-
)
31+
# log the operation level collective tracing information to a file
32+
comm_mode.log_comm_debug_tracing_table_to_file(
33+
noise_level=1, file_name="transformer_operation_log.txt"
34+
)
3535
36-
# dump the operation level collective tracing information to json file,
37-
# used in the visual browser below
38-
comm_mode.generate_json_dump(noise_level=2)
36+
# dump the operation level collective tracing information to json file,
37+
# used in the visual browser below
38+
comm_mode.generate_json_dump(noise_level=2)
3939
.. code-block:: python
40+
41+
All users have to do is wrap the code running the model in CommDebugMode and call the API that they want to use to display the data.
42+
Documentation Title
43+
===================
44+
45+
Introduction to the Module
46+
--------------------------
47+
48+
Below is the interactive module tree visualization:
49+
50+
.. raw:: html
51+
52+
<!DOCTYPE html>
53+
<html lang ="en">
54+
<head>
55+
<meta charset="UTF-8">
56+
<meta name = "viewport" content="width=device-width, initial-scale=1.0">
57+
<title>CommDebugMode Module Tree</title>
58+
<style>
59+
ul, #tree-container {
60+
list-style-type: none;
61+
margin: 0;
62+
padding: 0;
63+
}
64+
.caret {
65+
cursor: pointer;
66+
user-select: none;
67+
}
68+
.caret::before {
69+
content: "\25B6";
70+
color:black;
71+
display: inline-block;
72+
margin-right: 6px;
73+
}
74+
.caret-down::before {
75+
transform: rotate(90deg);
76+
}
77+
.tree {
78+
padding-left: 20px;
79+
}
80+
.tree ul {
81+
padding-left: 20px;
82+
}
83+
.nested {
84+
display: none;
85+
}
86+
.active {
87+
display: block;
88+
}
89+
.forward-pass,
90+
.backward-pass {
91+
margin-left: 40px;
92+
}
93+
.forward-pass table {
94+
margin-left: 40px;
95+
width: auto;
96+
}
97+
.forward-pass table td, .forward-pass table th {
98+
padding: 8px;
99+
}
100+
.forward-pass ul {
101+
display: none;
102+
}
103+
table {
104+
font-family: arial, sans-serif;
105+
border-collapse: collapse;
106+
width: 100%;
107+
}
108+
td, th {
109+
border: 1px solid #dddddd;
110+
text-align: left;
111+
padding: 8px;
112+
}
113+
tr:nth-child(even) {
114+
background-color: #dddddd;
115+
}
116+
#drop-area {
117+
position: relative;
118+
width: 25%;
119+
height: 100px;
120+
border: 2px dashed #ccc;
121+
border-radius: 5px;
122+
padding: 0px;
123+
text-align: center;
124+
}
125+
.drag-drop-block {
126+
display: inline-block;
127+
width: 200px;
128+
height: 50px;
129+
background-color: #f7f7f7;
130+
border: 1px solid #ccc;
131+
border-radius: 5px;
132+
padding: 10px;
133+
font-size: 14px;
134+
color: #666;
135+
cursor: pointer;
136+
}
137+
#file-input {
138+
position: absolute;
139+
top: 0;
140+
left: 0;
141+
width: 100%;
142+
height: 100%;
143+
opacity: 0;
144+
}
145+
</style>
146+
</head>
147+
<body>
148+
<div id="drop-area">
149+
<div class="drag-drop-block">
150+
<span>Drag file here</span>
151+
</div>
152+
<input type="file" id="file-input" accept=".json">
153+
</div>
154+
<div id="tree-container"></div>
155+
<script src="https://cdn.jsdelivr.net/gh/pytorch/pytorch@main/torch/distributed/_tensor/debug/comm_mode_broswer_visual.js"></script>
156+
</body>
157+
</html>
158+
159+
.. raw:: html

0 commit comments

Comments
 (0)