Skip to content

Add knowledge_distillation_tutorial #2514

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 33 commits into from
Aug 22, 2023
Merged
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
79b7467
Add knowledge_distillation_tutorial
AlexandrosChrtn Jul 31, 2023
7606376
Merge branch 'main' into knowledge-distillation-tutorial
Aug 7, 2023
137b313
Include description above and fix pyspelling
AlexandrosChrtn Aug 8, 2023
5618628
Merge branch 'main' into knowledge-distillation-tutorial
AlexandrosChrtn Aug 8, 2023
e260d4c
Merge branch 'main' into knowledge-distillation-tutorial
Aug 14, 2023
4f6eec8
Merge branch 'main' into knowledge-distillation-tutorial
AlexandrosChrtn Aug 15, 2023
a569ee9
review fixes + pyspelling
AlexandrosChrtn Aug 15, 2023
0632312
Merge branch 'main' into knowledge-distillation-tutorial
Aug 15, 2023
a5d8221
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
b7185df
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
86fa7a0
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
4f3570a
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
5881348
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
fbbb40d
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
9078639
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
f66b0ee
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
035c541
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
fd8b9b4
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
788aedf
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
de2d4ef
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
56b7002
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
f44a32d
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
1ced800
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
1113cf1
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
d166fff
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
196637a
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
1d32f4f
Update beginner_source/knowledge_distillation_tutorial.py
AlexandrosChrtn Aug 16, 2023
bccd00a
Apply suggestions from code review
AlexandrosChrtn Aug 17, 2023
abcdd60
review fixes
AlexandrosChrtn Aug 18, 2023
bf79878
review fixes
AlexandrosChrtn Aug 18, 2023
beec23c
review fixes
AlexandrosChrtn Aug 18, 2023
c79d298
final review fixes
AlexandrosChrtn Aug 18, 2023
cafd91e
Apply suggestions from code review
svekars Aug 22, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added _static/img/knowledge_distillation/ce_only.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
737 changes: 737 additions & 0 deletions beginner_source/knowledge_distillation_tutorial.py

Large diffs are not rendered by default.

8 changes: 8 additions & 0 deletions index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -585,6 +585,13 @@ What's new in PyTorch tutorials?
:image: _static/img/thumbnails/cropped/pytorch-logo.png
:link: intermediate/scaled_dot_product_attention_tutorial.html
:tags: Model-Optimization,Attention,Transformer

.. customcarditem::
:header: Knowledge Distillation in Convolutional Neural Networks
:card_description: Learn how to improve the accuracy of lightweight models using more powerful models as teachers.
:image: _static/img/thumbnails/cropped/knowledge_distillation_pytorch_logo.png
:link: beginner/knowledge_distillation_tutorial.html
:tags: Model-Optimization,Image/Video

.. Parallel-and-Distributed-Training

Expand Down Expand Up @@ -979,6 +986,7 @@ Additional Resources
intermediate/torch_compile_tutorial
intermediate/inductor_debug_cpu
intermediate/scaled_dot_product_attention_tutorial
beginner/knowledge_distillation_tutorial

.. toctree::
:maxdepth: 2
Expand Down