1
1
Distributed and Parallel Training Tutorials
2
2
===========================================
3
3
4
- This page includes all distributed and parallel trainings available
4
+ Distributed and
5
5
at pytorch.org website.
6
6
7
7
Getting Started with Distributed Data-Parallel Training (DDP)
8
8
-------------------------------------------------------------
9
9
10
+
11
+
10
12
.. grid :: 3
11
13
12
- .. grid-item-card :: Getting Started with PyTorch Distributed
14
+ .. grid-item-card :: :octicon:`file-code;1em`
15
+ Getting Started with PyTorch Distributed
13
16
:shadow: none
14
17
:link: https://example.com
15
18
:link-type: url
16
19
17
20
This tutorial provides a gentle intro to the PyTorch
18
21
DistributedData Parallel.
22
+
23
+ :octicon: `code;1em ` Code
19
24
20
- .. grid-item-card :: Single Machine Model Parallel Best Practices
25
+ .. grid-item-card :: :octicon:`file-code;1em`
26
+ Single Machine Model Parallel Best Practices
21
27
:shadow: none
22
28
:link: https://example.com
23
29
:link-type: url
24
30
25
31
In this tutorial you will learn about best practices in
26
32
using model parallel.
27
33
28
- .. grid-item-card :: Writing Distributed Applications with PyTorch
34
+ :octicon: `code;1em ` Code :octicon: `square-fill;1em ` :octicon: `video;1em ` Video
35
+
36
+ .. grid-item-card :: :octicon:`file-code;1em` Writing Distributed Applications with PyTorch
29
37
:shadow: none
30
38
:link: https://example.com
31
39
:link-type: url
32
40
33
41
This tutorial demonstrates how to write a distributed application
34
42
with PyTorch.
35
43
44
+ :octicon: `code;1em ` Code :octicon: `square-fill;1em ` :octicon: `video;1em ` Video
45
+
36
46
Learn FSDP
37
47
----------
38
48
@@ -43,33 +53,6 @@ models.
43
53
44
54
.. grid :: 3
45
55
46
- .. grid-item-card :: Getting Started with FSDP
47
- :shadow: none
48
- :img-top: ../_static/img/thumbnails/cropped/pytorch-logo.png
49
- :link: https://example.com
50
- :link-type: url
51
-
52
- This tutorial provides a gentle intro to the PyTorch
53
- DistributedData Parallel.
54
-
55
- .. grid-item-card :: Single Machine Model Parallel Best Practices
56
- :shadow: none
57
- :img-top: ../_static/img/thumbnails/cropped/pytorch-logo.png
58
- :link: https://example.com
59
- :link-type: url
60
-
61
- In this tutorial you will learn about best practices in
62
- using model parallel.
63
-
64
- .. grid-item-card :: Writing Distributed Applications with PyTorch
65
- :shadow: none
66
- :img-top: ../_static/img/thumbnails/cropped/pytorch-logo.png
67
- :link: https://example.com
68
- :link-type: url
69
-
70
- This tutorial demonstrates how to write a distributed application
71
- with PyTorch.
72
-
73
56
Learn RPC
74
57
---------
75
58
0 commit comments