Understanding the reason behind 'RuntimeError: leaf variable has been moved into the graph interior'

Question

I am trying to understand pytorch and how autograd works in it. I tried creating a tensor by filling it with values from other tensors and then checking the gradients. However, I am running into RuntimeError: leaf variable has been moved into the graph interior if I don't set requires_grad equal to False.

code:

x = torch.ones(3,5,requires_grad=True)

y = x+2

z = y*y*3

out1 = z.mean()
out2 = 2*z.mean()

outi = torch.empty(2,requires_grad=True)

outi[0] = out1
outi[1] = out2

outi.backward(torch.tensor([0.,1.]))

output:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
 in 
     13 outi[1] = out2
     14 
---> 15 outi.backward(torch.tensor([0.,1.]))

~/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/tensor.py in backward(self, gradient, retain_graph, create_graph)
    183                 products. Defaults to ``False``.
    184         """
--> 185         torch.autograd.backward(self, gradient, retain_graph, create_graph)
    186 
    187     def register_hook(self, hook):

~/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)
    123         retain_graph = create_graph
    124 
--> 125     Variable._execution_engine.run_backward(
    126         tensors, grad_tensors, retain_graph, create_graph,
    127         allow_unreachable=True)  # allow_unreachable flag

RuntimeError: leaf variable has been moved into the graph interior

however, I can change the requires_grad to False and it will all work just fine

x = torch.ones(3,5,requires_grad=True)

y = x+2

z = y*y*3

out1 = z.mean()
out2 = 2*z.mean()

outi = torch.empty(2,requires_grad=False)

outi[0] = out1
outi[1] = out2

outi.backward(torch.tensor([0.,1.]))

output:

empty. it worked

Can somebody please help me understand what happened under the hood and, what changed with setting require_grad to True that resulted in this behavior? Thank you for reading

user_10886420 · Answer

Intro

First, definition of what a leaf variable in PyTorch is, you can check official documentation for tensor.is_leaf (emphasis mine):

All Tensors that have requires_grad which is False will be leaf Tensors by convention.

For Tensors that have requires_grad which is True, they will be leaf Tensors if they were created by the user. This means that they are not the result of an operation and so grad_fn is None.

So let's see how this looks for outi variable in original code. Immediately after creation, running this snippet:

outi = torch.empty(2, requires_grad=True)
print(outi.is_leaf, outi.grad_fn, outi.requires_grad)

gives:

True, None, True

as it was created by user and there is no previous operation creating it so it should be the second bolded case from the above citation.

Now this line:

outi[0] = out1
outi[1] = out2

Uses two nodes which are not leafs and are part of the graph which goes back to x (which is the only leaf in it). By doing this outi is also part of the original x graph and would have to be backpropagated through, yet you specified it as a leaf (more on that later), which cannot be backpropagated through (by the definition they either don't require gradient or are created by user). Version of outi as leaf was already put on graph, after above assignment, this snippet:

print(outi.is_leaf, outi.grad_fn, outi.requires_grad)

changes to:

False  True

Error

Now, I agree it's a pretty uninformative error given that changing requires_grad=False does not make it non-leaf variable (requires_grad=False is implicit):

outi = torch.empty(2)
print(outi.is_leaf, outi.grad_fn, outi.requires_grad)
# > True None False

But this tensor could be "upgraded" to non-leaf tensor if you use assignment as you did without breaking the expected behaviour.

Why? Because you implicitly (or explicitly in case of your code) said you don't need gradient for this variable and PyTorch retains gradient only for leaf variables (unless you specify .retain_grad for specific tensor) due to memory optimization. So the only change here would be it will no longer be a leaf, but this would not break promises as .grad would be None anyway.

If you were to have requires_grad=True as you originally did you could, reasonably, according to PyTorch semantics, think that this:

outi.grad

Will give you a tensor with gradient. But if this requires_grad=True tensor were to be changed to non-leaf tensor, then, by definition it wouldn't have this field (as non-leaf tensors have .grad=None).

To me it seems like a design decision on their part to avoid confusion with requires_grad=True and breaking expected user experience.

BTW. If they were to disallow leaf variables inside graph then operation which works fine now (requires_grad=False) should be disallowed as well. But as requires_grad=False is implicit and often used (creating tensors or something like you did) it seems not to be to much of a stretch to allow it. Disallowing it would be much more severe. On the other hand if you specify requires_grad=True it could be assumed you know better what you are doing and really need that gradient here.

BTW2. This explanation might be a stretch but hopefully will shed some light. I haven't found anything official regarding this error (admittedly though I didn't dig too deep).

Some resources here, here (this one is important, someone was asking for justification of some design decisions though didn't get one AFAIK).

Comments

Comment 1

I think the requires_grad is getting inherited from the slice and also .grad is available.

Yes, it has requires_grad as True also as it's part of the graph now, BUT grad is not available as it is no longer a leaf. Printing outi.grad after backward gives you None and the following warning:

UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the gradient for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations.

So the .grad attribute is None anyway as user would expect giving requires_grad=False as creation argument. User could expect gradient to be not None if he was to set requires_grad=True and that's when PyTorch raises the error, IMO due to possible inconsistency with user expectation in that case.

Comment 2

For example:

a = torch.ones(2,requires_grad=False)
b = 2*a
b.requires_grad=True
print(b.is_leaf) #True

I have changed your code a little to go through it step by step:

a = torch.ones(2, requires_grad=False)
print(a.is_leaf) # True

We should start with a here, a is a leaf according to docs as:

All Tensors that have requires_grad which is False will be leaf Tensors by convention.

b = a * 2
print(b.is_leaf)

Now b is leaf as it does not require gradient (because a does not need a gradient it doesn't have to be backpropagated through this branch). Manipulating tensors with requires_grad=False creates tensors which do not require_grad otherwise it would be wasteful and non-sensical to turn it on.

b.requires_grad = True
print(b.is_leaf)

Now this one returns True as well. Once again, docs wording might not be the best here (as I've stated before), but (my additions in bold):

For Tensors that have requires_grad which is True (our case now) they will be leaf Tensors if they were created by the user (debatable about creation here as you have modified the existing one, indeed). This means that they are not the result of an operation and so grad_fn is None (this one IMO clarifies the previous point)

About clarification- as this tensor is not a result of any mathematical operation, you simply said you want this b tensor to require_grad. IMO it is a user created tensor as it was placed (created) on graph for the first time (before there was no need for that as it didn't require gradient).

And it does have it's requires_grad set to True, you did it explicitly here.

Comment 3 & 4

Everything with requires_grad=True is on the graph

Yes, but something with requires_grad=False can be on a graph as well if it is a leaf. Actually every PyTorch operation is created and added dynamically onto computational graph, here we use simplification: it's on graph if it takes part in backpropagation. For example neural network parameters are leafs yet they are on graph as the last part during backpropagation and they have their gradients (as they are optimized they need to be in graph in order to backprop through it).

Everything not on the graph is a leaf

Yes, essentially

everything on the graph that is not the product of operation on graph tensors is a leaf

Yes, if you add some tensor to it (e.g. created by torch.randn or such) is a leaf

every leaf on the graph and non-leaf where I set retain_grad=True manually will get .grad attribute populated.

Yes, it if it is part of backpropagation which is almost always the case in our "mental graph" case (I think at least). Unless it already has requires_grad=True, in this case it will be populated with gradient. Basically, except for creation, you shouldn't tinker with setting requires_grad=True as it is prone to fail (as you saw) and will definitely raise some eyebrows for other people reading your code.

every non-leaf on the graph has a grad_fn associated with it

Yes, that follows as some operation had to create it (and if it was created by some operation and this operation is differentiable, grad_fn is registered to be used during backward() call).