Update ddp_tutorial to work on Windows platform #1182
Conversation
update ddp_tutorial to work on Windows platform
|
Deploy preview for pytorch-tutorials-preview failed. Built with commit 1f0b816 https://app.netlify.com/sites/pytorch-tutorials-preview/deploys/5f9830fb6b616f0008a82ccb |
There was a problem hiding this comment.
LGTM!
@gunandrose4u would like you to add the following to the beginning of this doc:
Edited by: Joe Zhu <https://github.com/gunandrose4u>_
See below as an example:
intermediate_source/ddp_tutorial.rst
Outdated
| # initialize the process group | ||
| dist.init_process_group("gloo", rank=rank, world_size=world_size) | ||
| if sys.platform == 'win32': | ||
| # Distributed package only covers collective communications with Gloo backend, FileStore |
There was a problem hiding this comment.
nit: backend, -> backend and
intermediate_source/ddp_tutorial.rst
Outdated
| dist.init_process_group("gloo", rank=rank, world_size=world_size) | ||
| if sys.platform == 'win32': | ||
| # Distributed package only covers collective communications with Gloo backend, FileStore | ||
| # on Windows platform. Set init_method parameter in init_process_group and point it to a |
There was a problem hiding this comment.
nit: and point it to -> to
…4u/tutorials into jozh/ddp_win_tutorial
intermediate_source/ddp_tutorial.rst
Outdated
| # initialize the process group | ||
| dist.init_process_group("gloo", rank=rank, world_size=world_size) | ||
| if sys.platform == 'win32': | ||
| # Distributed package only covers collective communications with Gloo backend and FileStore |
There was a problem hiding this comment.
nit: below is how the built doc looks like. Shall we break the lines into shorter ones to avoid line wrapping?
# Distributed package only covers collective communications with
# Gloo backend and FileStore on Windows platform. Set init_method
# parameter in init_process_group to a local file.
# Example init_method="file:///f:/libtmp/some_file"
...
dist.init_process_group(
"gloo",
init_method=init_method,
rank=rank,
world_size=world_size
)
There was a problem hiding this comment.
Yes, we should wrap it. Just resolved.
|
@jlin27 this PR LGTM. Shall we land it (if tests pass, the current failure does not seem relevant). |
* Update ddp_tutorial.rst update ddp_tutorial to work on Windows platform * Update ddp tutorial to contain script for Windows and non-Windows platform together * Resolved code review comments * Wrap init_process_group method since it is too long in built doc * The comment is too long in one line, make it short. * Update Co-authored-by: Joe Zhu <jozh@microsoft.com> Co-authored-by: Brian Johnson <brianjo@fb.com>
Due to on Windows platform, DDP can only work with Gloo backend and FileStore. Update ddp_tutorial to clarify how to make ddp working on Windows platform.