Skip to content

Update links in RPC tutorial #946

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 23, 2021
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 13 additions & 13 deletions intermediate_source/rpc_tutorial.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ Prerequisites:
- `RPC API documents <https://pytorch.org/docs/master/rpc.html>`__

This tutorial uses two simple examples to demonstrate how to build distributed
training with the `torch.distributed.rpc <https://pytorch.org/docs/master/rpc.html>`__
package which is first introduced as a prototype feature in PyTorch v1.4.
training with the `torch.distributed.rpc <https://pytorch.org/docs/stable/rpc.html>`__
package which was first introduced as an experimental feature in PyTorch v1.4.
Source code of the two examples can be found in
`PyTorch examples <https://github.com/pytorch/examples>`__.

Expand All @@ -36,19 +36,19 @@ paradigms. For example:
machines.


The `torch.distributed.rpc <https://pytorch.org/docs/master/rpc.html>`__ package
can help with the above scenarios. In case 1, `RPC <https://pytorch.org/docs/master/rpc.html#rpc>`__
and `RRef <https://pytorch.org/docs/master/rpc.html#rref>`__ allow sending data
The `torch.distributed.rpc <https://pytorch.org/docs/stable/rpc.html>`__ package
can help with the above scenarios. In case 1, `RPC <https://pytorch.org/docs/stable/rpc.html#rpc>`__
and `RRef <https://pytorch.org/docs/stable/rpc.html#rref>`__ allow sending data
from one worker to another while easily referencing remote data objects. In
case 2, `distributed autograd <https://pytorch.org/docs/master/rpc.html#distributed-autograd-framework>`__
and `distributed optimizer <https://pytorch.org/docs/master/rpc.html#module-torch.distributed.optim>`__
case 2, `distributed autograd <https://pytorch.org/docs/stable/rpc.html#distributed-autograd-framework>`__
and `distributed optimizer <https://pytorch.org/docs/stable/rpc.html#module-torch.distributed.optim>`__
make executing backward pass and optimizer step as if it is local training. In
the next two sections, we will demonstrate APIs of
`torch.distributed.rpc <https://pytorch.org/docs/master/rpc.html>`__ using a
`torch.distributed.rpc <https://pytorch.org/docs/stable/rpc.html>`__ using a
reinforcement learning example and a language model example. Please note, this
tutorial does not aim at building the most accurate or efficient models to
solve given problems, instead, the main goal here is to show how to use the
`torch.distributed.rpc <https://pytorch.org/docs/master/rpc.html>`__ package to
`torch.distributed.rpc <https://pytorch.org/docs/stable/rpc.html>`__ package to
build distributed training applications.


Expand Down Expand Up @@ -289,10 +289,10 @@ observers. The agent serves as master by repeatedly calling ``run_episode`` and
``finish_episode`` until the running reward surpasses the reward threshold
specified by the environment. All observers passively waiting for commands
from the agent. The code is wrapped by
`rpc.init_rpc <https://pytorch.org/docs/master/rpc.html#torch.distributed.rpc.init_rpc>`__ and
`rpc.shutdown <https://pytorch.org/docs/master/rpc.html#torch.distributed.rpc.shutdown>`__,
`rpc.init_rpc <https://pytorch.org/docs/stable/rpc.html#torch.distributed.rpc.init_rpc>`__ and
`rpc.shutdown <https://pytorch.org/docs/stable/rpc.html#torch.distributed.rpc.shutdown>`__,
which initializes and terminates RPC instances respectively. More details are
available in the `API page <https://pytorch.org/docs/master/rpc.html>`__.
available in the `API page <https://pytorch.org/docs/stable/rpc.html>`__.


.. code:: python
Expand Down Expand Up @@ -442,7 +442,7 @@ takes a GPU tensor, you need to move it to the proper device explicitly.
With the above sub-modules, we can now piece them together using RPC to
create an RNN model. In the code below ``ps`` represents a parameter server,
which hosts parameters of the embedding table and the decoder. The constructor
uses the `remote <https://pytorch.org/docs/master/rpc.html#torch.distributed.rpc.remote>`__
uses the `remote <https://pytorch.org/docs/stable/rpc.html#torch.distributed.rpc.remote>`__
API to create an ``EmbeddingTable`` object and a ``Decoder`` object on the
parameter server, and locally creates the ``LSTM`` sub-module. During the
forward pass, the trainer uses the ``EmbeddingTable`` ``RRef`` to find the
Expand Down