diff --git a/intermediate_source/FSDP_adavnced_tutorial.rst b/intermediate_source/FSDP_adavnced_tutorial.rst index 748c8593306..5a0cb5376da 100644 --- a/intermediate_source/FSDP_adavnced_tutorial.rst +++ b/intermediate_source/FSDP_adavnced_tutorial.rst @@ -74,8 +74,8 @@ summarization using WikiHow dataset. The main focus of this tutorial is to highlight different available features in FSDP that are helpful for training large scale model above 3B parameters. Also, we cover specific features for Transformer based models. The code for this tutorial is available in `Pytorch -Examples -`__. +examples +`__. *Setup* @@ -97,13 +97,13 @@ Please create a `data` folder, download the WikiHow dataset from `wikihowAll.csv `wikihowSep.cs `__, and place them in the `data` folder. We will use the wikihow dataset from `summarization_dataset -`__. +`__. Next, we add the following code snippets to a Python script “T5_training.py”. .. note:: The full source code for this tutorial is available in `PyTorch examples - `__. + `__. 1.3 Import necessary packages: