Remove references to non-`gloo` backends in "Writing Distributed Applications with PyTorch" tutorial

[SethHWeidman]

Hey @pietern, @mrshenli,

The Writing Distributed Applications with PyTorch tutorial does not run as written with either tcp or mpi backends. With gloo, it works.

Can you please replace code snippets that reference tcp, such as this one:

dist.init_process_group(init_method='tcp://10.1.1.20:23456', rank=args.rank, world_size=4)

with the appropriate ones that reference gloo? Even better, if you could remove the references to TCP and MPI throughout the tutorial, that would be great.

Seems I'm not the only one to notice this; see #618 .

Thanks, and let me know if I can help!

[pietern]

Thanks for flagging this, @SethHWeidman!

There is a difference between the tcp initialization method and the tcp backend. The latter was deprecated in 1.0 and should indeed be replaced. The initialization method, however, is still usable, and many times the preferred way to initialize a process group.

The statement you included in your post will work just fine, seeing as the backend kwarg will default to gloo there. Only the parts where we say backend="tcp" should be changed.

[SethHWeidman]

Thanks, @pietern.

Could you or Shen take a stab at revising this tutorial to remove all references to the deprecated backends "MPI" and "TCP"? There are currently entire sections where we discuss the MPI and TCP backends. It might simply be a matter of deleting some text, but there are also places where we refer to "multiple backends", so could we change those to "Gloo" or something appropriate?

I took a stab at this myself in a prior PR, but quickly realized I knew nothing about these backends and so wasn't sure if I was revising things correctly. Really appreciate this cleanup work!

[pietern]

@SethHWeidman @mrshenli Taking a pass now.

[pietern]

@SethHWeidman Please check out #678. This should bring it back up to date.