Skip to content

Commit 52994ee

Browse files
committed
tweaks
Signed-off-by: Chris Abraham <cjyabraham@gmail.com>
1 parent 42faa38 commit 52994ee

File tree

1 file changed

+3
-5
lines changed

1 file changed

+3
-5
lines changed

_posts/2024-07-08-accelerated-pytorch-inference.md

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -240,15 +240,13 @@ Self CPU time total: 14.379ms
240240

241241
Underpinning torch.compile are new technologies – TorchDynamo, AOTDispatcher, and TorchInductor.
242242

243-
**TorchDynamo** captures PyTorch programs safely using Python Frame Evaluation Hooks
244-
245-
**AOTDispatcher** overloads PyTorch’s autograd engine as a tracing autodiff for generating ahead-of-time backward traces.
246-
243+
**TorchDynamo** captures PyTorch programs safely using Python Frame Evaluation Hooks
244+
**AOTDispatcher** overloads PyTorch’s autograd engine as a tracing autodiff for generating ahead-of-time backward traces.
247245
**TorchInductor** is a deep learning compiler that generates fast code for multiple accelerators and backends.
248246

249247
![The PyTorch compilation process source](/assets/images/accelerated-pytorch-inference/fg3.png){:style="width:100%"}
250248

251-
_**Image 3**: The PyTorch compilation process; source: [https://pytorch.org/get-started/pytorch-2.0/](https://pytorch.org/get-started/pytorch-2.0/)_
249+
_**Image 3**: The PyTorch compilation process_
252250

253251
When torch.compile is invoked, torch dynamo rewrites Python bytecode to extract sequences of PyTorch operations into an [FX](https://pytorch.org/docs/stable/fx.html) [Graph](https://pytorch.org/docs/stable/fx.html), which is then compiled with inductor backend. For a typical inference scenario where the graph is frozen and gradient calculations are disabled, the inductor invokes platform specific optimizations like graph rewrite into more performant operators, operator fusion, and weights pre-packing.
254252

0 commit comments

Comments
 (0)