@@ -12,7 +12,7 @@ There are a few ways you can perform distributed training in
12
12
PyTorch with each method having their advantages in certain use cases:
13
13
14
14
* `DistributedDataParallel (DDP) <#learn-ddp >`__
15
- * `Fully Sharded Data Parallel (FSDP ) <#learn-fsdp >`__
15
+ * `Fully Sharded Data Parallel (FSDP2 ) <#learn-fsdp >`__
16
16
* `Tensor Parallel (TP) <#learn-tp >`__
17
17
* `Device Mesh <#device-mesh >`__
18
18
* `Remote Procedure Call (RPC) distributed training <#learn-rpc >`__
@@ -60,28 +60,18 @@ Learn DDP
60
60
61
61
.. _learn-fsdp :
62
62
63
- Learn FSDP
63
+ Learn FSDP2
64
64
----------
65
65
66
66
.. grid :: 3
67
67
68
68
.. grid-item-card :: :octicon:`file-code;1em`
69
- Getting Started with FSDP
69
+ Getting Started with FSDP2
70
70
:link: https://pytorch.org/tutorials/intermediate/FSDP_tutorial.html?utm_source=distr_landing&utm_medium=FSDP_getting_started
71
71
:link-type: url
72
72
73
73
This tutorial demonstrates how you can perform distributed training
74
- with FSDP on a MNIST dataset.
75
- +++
76
- :octicon: `code;1em ` Code
77
-
78
- .. grid-item-card :: :octicon:`file-code;1em`
79
- FSDP Advanced
80
- :link: https://pytorch.org/tutorials/intermediate/FSDP_advanced_tutorial.html?utm_source=distr_landing&utm_medium=FSDP_advanced
81
- :link-type: url
82
-
83
- In this tutorial, you will learn how to fine-tune a HuggingFace (HF) T5
84
- model with FSDP for text summarization.
74
+ with FSDP2 on a transformer model
85
75
+++
86
76
:octicon: `code;1em ` Code
87
77
@@ -196,7 +186,6 @@ Custom Extensions
196
186
intermediate/ddp_tutorial
197
187
intermediate/dist_tuto
198
188
intermediate/FSDP_tutorial
199
- intermediate/FSDP_advanced_tutorial
200
189
intermediate/TCPStore_libuv_backend
201
190
intermediate/TP_tutorial
202
191
intermediate/pipelining_tutorial
0 commit comments