Add a tutorial for Tensorboard profiler #1380
Conversation
|
Deploy preview for pytorch-tutorials-preview ready! Built with commit 939474e https://deploy-preview-1380--pytorch-tutorials-preview.netlify.app |
gaoteng-git
force-pushed
the
tensorboard_profiler
branch
from
5efecaf to
86fcc26
Compare
gaoteng-git
force-pushed
the
tensorboard_profiler
branch
from
0dac772 to
4868a5e
Compare
gaoteng-git
force-pushed
the
tensorboard_profiler
branch
from
137c71c to
4919e6d
Compare
gaoteng-git
force-pushed
the
tensorboard_profiler
branch
3 times, most recently
from
4d8f9ef to
c2d8223
Compare
gaoteng-git
force-pushed
the
tensorboard_profiler
branch
from
c2d8223 to
32593d0
Compare
|
@gaoteng-git would the recipe for deprecated profiler be updated too? |
I think it's better to add deprecation comment and link to this new tutorial. Only after this new tutorial is merged, the link will be available. |
| This recipe demonstrates how to use PyTorch Profiler | ||
| to detect performance bottlenecks of the model. |
There was a problem hiding this comment.
let's say that this tutorial demonstrates how to use TensorBoard plugin with PyTorch Profiler to detect performance bottlenecks of the model
| This recipe demonstrates how to use PyTorch Profiler | ||
| to detect performance bottlenecks of the model. | ||
|
|
||
| .. note:: |
There was a problem hiding this comment.
I don't think we need this note, since your tutorial already uses new API, we don't need to highlight the older vs newer API here, users would just use the new API (according to this tutorial) and it's fine
| The profiler can visualize this information | ||
| in TensorBoard Plugin and provide analysis of the performance bottlenecks. | ||
|
|
||
| In this recipe, we will use a simple Resnet model to demonstrate how to |
There was a problem hiding this comment.
In this tutorial ... how to use TensorBoard plugin to analyze model performance
| # During ``warmup`` steps, the profiler starts profiling as warmup but does not record any events. | ||
| # This is for reducing the profiling overhead. | ||
| # The overhead at the beginning of profiling is high and easy to bring skew to the profiling result. | ||
| # During ``active`` steps, the profiler works and record events. |
| # During ``active`` steps, the profiler works and record events. | ||
| # - ``on_trace_ready`` - callable that is called at the end of each cycle; | ||
| # In this example we use ``torch.profiler.tensorboard_trace_handler`` to generate result files for TensorBoard. | ||
| # After profiling, result files will be saved into the ``./log/resnet18`` directory. |
There was a problem hiding this comment.
into the directory specified with logdir parameter
| # | ||
| # - ``schedule`` - callable that takes step (int) as a single parameter | ||
| # and returns the profiler action to perform at each step; | ||
| # In this example with wait=1, warmup=1, active=5, |
There was a problem hiding this comment.
let's also use the code quotes around wait=1, warmup=1, active=5
| if step >= 7: | ||
| break | ||
| train(batch_data) | ||
| prof.step() |
There was a problem hiding this comment.
let's add a comment here saying that we call prof.step() after each step to notify the profiler
| # .. image:: ../../_static/img/profiler_trace_view2.png | ||
| # :scale: 25 % | ||
| # | ||
| # From the above view, we can find the event of ``enumerate(DataLoader)`` is shortened, |
There was a problem hiding this comment.
From the view above, we can see that the runtime of enumerate(DataLoader) is reduced
|
Looks really good, thank you! |
Co-authored-by: maxluk <maxluk@microsoft.com>
gaoteng-git
force-pushed
the
tensorboard_profiler
branch
from
80f8332 to
939474e
Compare
* add tensorboard_profiler tutorial * Update intermediate_source/tensorboard_profiler_tutorial.py Co-authored-by: maxluk <maxluk@microsoft.com> * update title * remove testing on windows because kineto doesn't support windows now * rename * Update with the help of Ilia Co-authored-by: Teng Gao <tegao@microsoft.com> Co-authored-by: maxluk <maxluk@microsoft.com>
@brianjo @chauhang @ilia-cher @gdankel This is a tutorial introducing usage of kineto-based profiler and visualization in TensorBoard. Would you please help merge it?