|
227 | 227 | # normalization layers to evaluation mode before running inference.
|
228 | 228 | # Failing to do this will yield inconsistent inference results.
|
229 | 229 | #
|
| 230 | +# Export/Load Model in TorchScript Format |
| 231 | +# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 232 | +# |
| 233 | +# One common way to do inference with a trained model is to use |
| 234 | +# `TorchScript <https://pytorch.org/docs/stable/jit.html>`__, an intermediate |
| 235 | +# representation of a PyTorch model that can be run in Python as well as in a |
| 236 | +# high performance environment like C++. TorchScript is actually the recommended model format |
| 237 | +# for scaled inference and deployment. |
| 238 | +# |
| 239 | +# .. note:: |
| 240 | +# Using the TorchScript format, you will be able to load the exported model and |
| 241 | +# run inference without defining the model class. |
| 242 | +# |
| 243 | +# **Export:** |
| 244 | +# |
| 245 | +# .. code:: python |
| 246 | +# |
| 247 | +# model_scripted = torch.jit.script(model) # Export to TorchScript |
| 248 | +# model_scripted.save('model_scripted.pt') # Save |
| 249 | +# |
| 250 | +# **Load:** |
| 251 | +# |
| 252 | +# .. code:: python |
| 253 | +# |
| 254 | +# model = torch.jit.load('model_scripted.pt') |
| 255 | +# model.eval() |
| 256 | +# |
| 257 | +# Remember that you must call ``model.eval()`` to set dropout and batch |
| 258 | +# normalization layers to evaluation mode before running inference. |
| 259 | +# Failing to do this will yield inconsistent inference results. |
| 260 | +# |
| 261 | +# For more information on TorchScript, feel free to visit the dedicated |
| 262 | +# `tutorials <https://pytorch.org/tutorials/beginner/Intro_to_TorchScript_tutorial.html>`__. |
| 263 | +# You will get familiar with the tracing conversion and learn how to |
| 264 | +# run a TorchScript module in a `C++ environment <https://pytorch.org/tutorials/advanced/cpp_export.html>`__. |
| 265 | + |
230 | 266 |
|
231 | 267 |
|
232 | 268 | ######################################################################
|
|
0 commit comments