The 'subspace of an original dimension' dilemma

In order to use tensor.view() the tensor must satisfy two conditions- each new view dimension must either be a

1. subspace of an original dimension
2. or only span across original dimensions ...

Lets discuss this one by one,

First, regarding subspace of an original dimension you need to understand the concept behind subspace. Not going into mathematical detail but in short - subspace is a subset of infinite number of n dimensional vectors(Rn) and the vectors inside the subspace must follow 2 rules -

i) Sub space will contain the Zero vector(0n)

ii) Must satisfy closure under Multiplication and addition

To visualise this in mind you can consider a 2D plane containing infinite lines. So the subspace of that 2D vector space will be set of lines which will pass through origin. These lines satisfies above two conditions.

Now there is a concept called projection of subspaces. Without digging into too much mathematical detail you can consider it as a regular line projection but for subspaces.

Now back to the point, lets assume if you have a tensor of size (4,5), you can actually consider it as a 20 dimensional vector. Assume you have a 20D space, and the subspaces will pass through the origin, and if you want to make projection of any line l1 from subspace with respect to any 2 axes, tensor.view() will output projection of that line with (2,10).

For the second part, you need to understand the concept of contiguous and non-contiguous memory allocation in pytorch. As its out of scope of the question I am going to explain it in very brief. If you view a n dimensional vector as (n/k, k) and if you run tensor.stride() on the new vector it will show you the stride for memory allocation in x and y direction. Now if you run view() again with different dimensions then this following equation must hold true for successful conversion due to non-contiguous memory allocation.

I tried my best to explain it in brief, let me know if you have more questions.

After some thought, I think the sentence could be interpreted as follows (despite not being mathematically formal).

Suppose to have a tensor t of size (4, 6), that can be seen as an ordered set of four 6d row vectors residing in a vector space.

We can perform a tensor view v = t.view(4, 2, 3). v has now two new view dimensions: 2 and 3. We can see it as an ordered set containing four ordered sets of two 3d vectors, or three 2d vectors, depending on how we consider them.

Such new, smaller vectors can be mathematically seen as projections of the original 6-element vectors onto a vector subspace.