Do not use deepcopy to copy nn.modules (including entire models)

[mikekgfb]

copy.deepcopy is not defined for nn.module() and does not reliably copy an nn.module hierrchy, such as a model or partial model. Our tutorials should using copy.deepcopy() as this will induce our users to make incorrect use of the primitive.

Please update https://pytorch.org/tutorials/beginner/transformer_tutorial.html and other tutorials to avoid the use. Recommended way for snapshotting is to use torch.save() for trained models. to create multiple clones of an untrained model, create multiple copies from first principles from the model architectural parameters.

cc @pytorch/team-text-core @Nayef211

[mikekgfb]

cc: @mthrok

[Nayef211]

Thanks for creating this issue @mikekgfb. Will submit a PR with the fixes soon.

[Nayef211]

@svekars I don't think this issue should be closed. The PR I sent out fixed the issue for https://pytorch.org/tutorials/beginner/transformer_tutorial.html. But there are several other places where it's still being used.

I.e. https://github.com/search?q=repo%3Apytorch%2Ftutorials%20copy.deepcopy(&type=code

[svekars]

Can the owners of the tutorials please take a look and submit corrections? If some of these tutorials describe an obsolete feature, please submit a PR to remove them:

• fx_numeric_suite_tutorial - will not fix, will remove the tutorial as no longer valid.
• fx_graph_mode_ptq_static - @jerryzh168
• fx_conv_bn_fuser - @Chillee
• transfer_learning_tutorial - @z-a-f
• quantized_transfer_learning_tutorial - @z-a-f
• fx_graph_mode_ptq_dynamic - @jerryzh168
• mario_rl - @suraj813 @hw26

[vkuzo]

For quantization, we were assuming that copy.deepcopy is defined on nn.Module. Is it really not defined - are we sure that it's not just a bug that it's not working in some cases?

[jerryzh168]

cc @qihqi

[qihqi]

Q: copy.deepcopy is not defined for nn.module() and does not reliably copy an nn.module hierrchy -- I believe in truthiness of this statement. However, why is the CTA "stop using deepcopy" and not "let's fix deepcopy for nn.Module"? Like is there a fundamental reason the latter is impossible or undesired, or it's really, "until deepcopy start working properly don't use it"?

[chedatomasz]

Are you sure deepcopy is not supposed to be defined for nn.modules? https://github.com/pytorch/pytorch/blob/master/torch/optim/swa_utils.py appears to be using deepcopy.