Computing matrix derivatives with torch.autograd.grad (PyTorch)

user_3262154 0
Viewed 924

I am trying to compute matrix derivatives in PyTorch using torch.autograd.grad however I am running into few issues. Here is a minimal working example to reproduce the error.

theta = torch.tensor(np.random.uniform(low=-np.pi, high=np.pi), requires_grad=True)
rot_mat = torch.tensor([[torch.cos(theta), torch.sin(theta), 0], 
                        [-torch.sin(theta), torch.cos(theta), 0]], 
                        dtype=torch.float, requires_grad=True)
torch.autograd.grad(outputs=rot_mat, 
                    inputs=theta, grad_outputs=torch.ones_like(rot_mat), 
                    create_graph=True, retain_graph=True)

This code results in the error "One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior."

I tried using allow_unused=True but the gradients are returned as None. I am not sure what is causing the graph to be disconnected here.

autograd pytorch
1 Answers

Pytorch autograd graph will be created only if pytorch functions are used.

I think python 2d list used while creating rot_mat disconnects the graph. So using torch functions create rotation matrix and also just use backward() function to compute gradients. Here's sample code:

import torch
import numpy as np

theta   = torch.tensor(np.random.uniform(low=-np.pi, high=np.pi), requires_grad=True)

# create required values and convert it to torch 1d tensor
cos_t   = torch.cos(theta).view(1)
sin_t   = torch.sin(theta).view(1)
msin_t  = -sin_t
zero    = torch.zeros(1)

# create rotation matrix using only pytorch functions
rot_1d  = torch.cat((cos_t, sin_t, zero, msin_t, cos_t, zero))
rot_mat = rot_1d.view((2, 3)) 

# Autograd
rot_mat.backward(torch.ones_like(rot_mat))

# gradient
print(theta.grad)
user_141087340
Related