Fix spellcheck (#2713)

* Fix spellcheck
* Update .pyspelling.yml to ignore references

Author: svekars

.pyspelling.yml

@@ -45,6 +45,9 @@ matrix:
         - open: '\.\. (code-block|math)::.*$\n*'
           content: '(?P<first>(^(?P<indent>[ ]+).*$\n))(?P<other>(^([ \t]+.*|[ \t]*)$\n)*)'
           close: '(^(?![ \t]+.*$))'
+        # Ignore references like "[1] Author: Title"
+        - open: '\[\d\]'
+          close: '\n'
   - pyspelling.filters.markdown:
   - pyspelling.filters.html:
       ignores:

advanced_source/usb_semisup_learn.py

@@ -1,70 +1,88 @@
 """
 Semi-Supervised Learning using USB built upon PyTorch
-=============================
-
+=====================================================
 
 **Author**: `Hao Chen <https://github.com/Hhhhhhao>`_
- 
-
-Introduction
-------------
 
-USB is a semi-supervised learning framework built upon PyTorch.
-Based on Datasets and Modules provided by PyTorch, USB becomes a flexible, modular, and easy-to-use framework for semi-supervised learning.
-It supports a variety of semi-supervised learning algorithms, including FixMatch, FreeMatch, DeFixMatch, SoftMatch, etc.
+Unified Semi-supervised learning Benchmark (USB) is a semi-supervised
+learning framework built upon PyTorch.
+Based on Datasets and Modules provided by PyTorch, USB becomes a flexible,
+modular, and easy-to-use framework for semi-supervised learning.
+It supports a variety of semi-supervised learning algorithms, including
+``FixMatch``, ``FreeMatch``, ``DeFixMatch``, ``SoftMatch``, and so on.
 It also supports a variety of imbalanced semi-supervised learning algorithms.
-The benchmark results across different datasets of computer vision, natural language processing, and speech processing are included in USB.
+The benchmark results across different datasets of computer vision, natural
+language processing, and speech processing are included in USB.
+
+This tutorial will walk you through the basics of using the USB lighting
+package.
+Let's get started by training a ``FreeMatch``/``SoftMatch`` model on
+CIFAR-10 using pretrained ViT!
+And we will show it is easy to change the semi-supervised algorithm and train
+on imbalanced datasets.
 
-This tutorial will walk you through the basics of using the usb lighting package. 
-Let's get started by training a FreeMatch/SoftMatch model on CIFAR-10 using pre-trained ViT!
-And we will show it is easy to change the semi-supervised algorithm and train on imbalanced datasets.
 
- 
 .. figure:: /_static/img/usb_semisup_learn/code.png
    :alt: USB framework illustration
 """
 
 
 ######################################################################
-# Introduction to FreeMatch and SoftMatch in Semi-Supervised Learning
-# --------------------
-# Here we provide a brief introduction to FreeMatch and SoftMatch.
-# First we introduce a famous baseline for semi-supervised learning called FixMatch.
-# FixMatch is a very simple framework for semi-supervised learning, where it utilizes a strong augmentation to generate pseudo labels for unlabeled data.
-# It adopts a confidence thresholding strategy to filter out the low-confidence pseudo labels with a fixed threshold set.
-# FreeMatch and SoftMatch are two algorithms that improve upon FixMatch.
-# FreeMatch proposes adaptive thresholding strategy to replace the fixed thresholding strategy in FixMatch. 
-# The adaptive thresholding progressively increases the threshold according to the learning status of the model on each class.
-# SoftMatch absorbs the idea of confidence thresholding as an weighting mechanism.
-# It proposes a Gaussian weighting mechanism to overcome the quantity-quality trade-off in pseudo-labels. 
-# In this tutorial, we will use USB to train FreeMatch and SoftMatch.
+# Introduction to ``FreeMatch`` and ``SoftMatch`` in Semi-Supervised Learning
+# ---------------------------------------------------------------------------
+#
+# Here we provide a brief introduction to ``FreeMatch`` and ``SoftMatch``.
+# First, we introduce a famous baseline for semi-supervised learning called ``FixMatch``.
+# ``FixMatch`` is a very simple framework for semi-supervised learning, where it
+# utilizes a strong augmentation to generate pseudo labels for unlabeled data.
+# It adopts a confidence thresholding strategy to filter out the low-confidence
+# pseudo labels with a fixed threshold set.
+# ``FreeMatch`` and ``SoftMatch`` are two algorithms that improve upon ``FixMatch``.
+# ``FreeMatch`` proposes adaptive thresholding strategy to replace the fixed
+# thresholding strategy in ``FixMatch``. The adaptive thresholding progressively
+# increases the threshold according to the learning status of the model on each
+# class. ``SoftMatch`` absorbs the idea of confidence thresholding as an
+# weighting mechanism. It proposes a Gaussian weighting mechanism to overcome
+# the quantity-quality trade-off in pseudo-labels. In this tutorial, we will
+# use USB to train ``FreeMatch`` and ``SoftMatch``.
 
 
 ######################################################################
-# Use USB to Train FreeMatch/SoftMatch on CIFAR-10 with only 40 labels
-# --------------------
-# USB is a Pytorch-based Python package for Semi-Supervised Learning (SSL). 
-# It is easy-to-use/extend, affordable to small groups, and comprehensive for developing and evaluating SSL algorithms. 
-# USB provides the implementation of 14 SSL algorithms based on Consistency Regularization, and 15 tasks for evaluation from CV, NLP, and Audio domain.
-# It has a modular design that allows users to easily extend the package by adding new algorithms and tasks.
-# It also supports a python api for easier adaptation to different SSL algorithms on new data.
-# 
-# 
-# Now, let's use USB to train FreeMatch and SoftMatch on CIFAR-10.
-# First, we need to install USB package ``semilearn`` and import necessary api functions from USB.
+# Use USB to Train ``FreeMatch``/``SoftMatch`` on CIFAR-10 with only 40 labels
+# ----------------------------------------------------------------------------
+#
+# USB is easy to use and extend, affordable to small groups, and comprehensive
+# for developing and evaluating SSL algorithms.
+# USB provides the implementation of 14 SSL algorithms based on Consistency
+# Regularization, and 15 tasks for evaluation from CV, NLP, and Audio domain.
+# It has a modular design that allows users to easily extend the package by
+# adding new algorithms and tasks.
+# It also supports a Python API for easier adaptation to different SSL
+# algorithms on new data.
+#
+#
+# Now, let's use USB to train ``FreeMatch`` and ``SoftMatch`` on CIFAR-10.
+# First, we need to install USB package ``semilearn`` and import necessary API
+# functions from USB.
 # Below is a list of functions we will use from ``semilearn``:
+#
 # - ``get_dataset`` to load dataset, here we use CIFAR-10
-# - ``get_data_loader`` to create train (labeled and unlabeled) and test data loaders, the train unlabeled loaders will provide both strong and weak augmentation of unlabeled data
-# - ``get_net_builder`` to create a model, here we use pre-trained ViT
-# - ``get_algorithm`` to create the semi-supervised learning algorithm, here we use FreeMatch and SoftMatch
+# - ``get_data_loader`` to create train (labeled and unlabeled) and test data
+# loaders, the train unlabeled loaders will provide both strong and weak
+# augmentation of unlabeled data
+# - ``get_net_builder`` to create a model, here we use pretrained ViT
+# - ``get_algorithm`` to create the semi-supervised learning algorithm,
+# here we use ``FreeMatch`` and ``SoftMatch``
 # - ``get_config``: to get default configuration of the algorithm
-# - ``Trainer``: a Trainer class for training and evaluating the algorithm on dataset
+# - ``Trainer``: a Trainer class for training and evaluating the
+# algorithm on dataset
 # 
 import semilearn
 from semilearn import get_dataset, get_data_loader, get_net_builder, get_algorithm, get_config, Trainer
 
 ######################################################################
-# After importing necessary functions, we first set the hyper-parameters of the algorithm.
+# After importing necessary functions, we first set the hyper-parameters of the
+# algorithm.
 # 
 config = {
     'algorithm': 'freematch',
@@ -122,27 +140,31 @@
 
 
 ######################################################################
-# We can start Train the algorithms on CIFAR-10 with 40 labels now.
+# We can start training the algorithms on CIFAR-10 with 40 labels now.
 # We train for 4000 iterations and evaluate every 500 iterations.
 # 
 trainer = Trainer(config, algorithm)
 trainer.fit(train_lb_loader, train_ulb_loader, eval_loader)
 
 
 ######################################################################
-# Finally, let's evaluate the trained model on validation set.
-# After training 4000 iterations with FreeMatch on only 40 labels of CIFAR-10, we obtain a classifier that achieves above 93 accuracy on validation set.
+# Finally, let's evaluate the trained model on the validation set.
+# After training 4000 iterations with ``FreeMatch`` on only 40 labels of
+# CIFAR-10, we obtain a classifier that achieves above 93 accuracy on the validation set.
 trainer.evaluate(eval_loader)
 
 
 
 ######################################################################
-# Use USB to Train SoftMatch with specific imbalanced algorithm on imbalanced CIFAR-10
-# --------------------
+# Use USB to Train ``SoftMatch`` with specific imbalanced algorithm on imbalanced CIFAR-10
+# ------------------------------------------------------------------------------------
 # 
-# Now let's say we have imbalanced labeled set and unlabeled set of CIFAR-10, and we want to train a SoftMatch model on it.
-# We create an imbalanced labeled set and imbalanced unlabeled set of CIFAR-10, by setting the ``lb_imb_ratio`` and ``ulb_imb_ratio`` to 10.
-# Also we replace the ``algorithm`` with ``softmatch`` and set the ``imbalanced`` to ``True``.
+# Now let's say we have imbalanced labeled set and unlabeled set of CIFAR-10,
+# and we want to train a ``SoftMatch`` model on it.
+# We create an imbalanced labeled set and imbalanced unlabeled set of CIFAR-10,
+# by setting the ``lb_imb_ratio`` and ``ulb_imb_ratio`` to 10.
+# Also, we replace the ``algorithm`` with ``softmatch`` and set the ``imbalanced``
+# to ``True``.
 # 
 config = {
     'algorithm': 'softmatch',
@@ -210,7 +232,7 @@
 
 
 ######################################################################
-# Finally, let's evaluate the trained model on validation set.
+# Finally, let's evaluate the trained model on the validation set.
 # 
 trainer.evaluate(eval_loader)
 

en-wordlist.txt

@@ -1,3 +1,9 @@
+SSL
+ViT
+Hao
+Chen
+Yidong
+Wang
 NDK
 Backpropagating
 multinode
@@ -124,6 +130,7 @@ JSON
 JVP
 Jacobian
 Kiuk
+Kihyuk
 Kubernetes
 Kuei
 LSTM
@@ -195,6 +202,7 @@ SciPy
 Sequentials
 Sigmoid
 SoTA
+Sohn
 Spacy
 TPU
 TensorBoard