I'm trying to understand how out= parameters work in numpy, pytorch and similar libraries.
Intuitively I would expect it to allow allocation free arithmetic. That is, where C = A @ B would allocate a new matrix, np.matmul(A, B, out=C) would use the space already allocated for C, thus requiring less memory management and faster code.
What confuses me is that the code
np.matmul(A, B, out=A) seems to work correctly.
That is, it computes A = A @ B without error.
So if this is really done without allocating new memory, then numpy (or pytorch) must be using some sort of in-place matrix multiplication algorithm. But I don't think those really exist? (See e.g. Is there an algorithm to multiply square matrices in-place?)
So am I wrong that out= doesn't allocate memory? Or is it just that numpy and pytorch are smart enough to realize that the output is the same as one of the inputs, and then allocates memory in this particular case?
I tried reading the documentation (https://pytorch.org/docs/stable/generated/torch.matmul.html) but it doesn't give any details of what's really going on.