Revisit all *_out ops

[pinzhenx]

Pytorch has a legacy resizing behavior in *_out ops (see here) in which out= tensor will only reuse the underlying buffer (if it's large enough, or new buffer will be allocated) regardless of its metadata. And the metadata will be rewritten by the op itself.

import torch

o = torch.rand(42)
print(o.size())
print(o.data_ptr())

torch.add(torch.rand(3, 8), torch.rand(3, 8), out=o)
print(o.size())
print(o.data_ptr())

torch.add(torch.rand(43), torch.rand(43), out=o)
print(o.size())
print(o.data_ptr())

torch.Size([42])
93863984799808
torch.Size([3, 8])
93863984799808   # 3 x 8 < 42, reuse old buffer
torch.Size([43])
93864012230144   # 43 > 42, allocate new buffer

[pinzhenx]

One possible solution is to infer output shape in DevOps and perform necessary resizing there