How to assign particular value to net.parameters() using torch

user_15217406 0
Viewed 864

This is my network; I have 2 CNN layers and 1 FC layer.

I've already trained CNN filters for particular domain, so I'm trying to train the FC layer only.

class Net(nn.Module): 
    def __init__(self):
        super(Net, self).__init__()

        self.layer1 = nn.Sequential(
            nn.Conv2d(in_channels = 1, out_channels = 1, kernel_size = (2,2), stride = 1, padding = 1, bias = False),
            nn.AvgPool2d(kernel_size = (2,2), stride = (2,2)),
            )
        self.layer2 = nn.Sequential(
            nn.Conv2d(in_channels = 1, out_channels = 1, kernel_size = (2,2), stride = 1, padding = 1, bias = False),
            nn.AvgPool2d(kernel_size = (2,2), stride = (2,2)),
            )
        self.fc = nn.Linear(1200, 1)
        self.dropout = nn.Dropout(p = 0.10)
        self.sigmoid = nn.Sigmoid()
        
            
    def forward(self, x):
        out1 = self.layer1(x)
        out2 = self.layer2(out1)
        out3 = out2.reshape(out2.size(0), -1)
        output = self.sigmoid(self.dropout((self.fc(out3))))
        return output

And I am trying to assign pre-trained values of parameters in this network.

params = list(Net().parameters())
params

Output:

[Parameter containing:
 tensor([[[[ 0.2240,  0.2135],
           [-0.2901,  0.4827]]]], requires_grad=True),
 Parameter containing:
 tensor([[[[-0.0363, -0.2801],
           [ 0.0853, -0.0217]]]], requires_grad=True),
 Parameter containing:
 tensor([[-0.0155, -0.0073, -0.0065,  ...,  0.0012, -0.0213, -0.0287]],
        requires_grad=True),
 Parameter containing:
 tensor([-0.0010], requires_grad=True)]


params[0][0][0][0][0] = -0.2454

RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation.

Is there any specific way to assign values to network's parameters?

tensor pytorch
1 Answers

Parameters by default have requires_grad=True, as you can see from the print of params. Runtime error points to that, meaning you can only modify your parameters in-place when they don't need to calculate gradients. One easy way for that is to use no_grad():

with torch.no_grad():
  params[0][0][0][0][0] = -0.2454

This will let you modify your parameters in-place. You can also change whole parameter at once without using no_grad. For example:

net = Net()
weight = net.layer1[0].weight          # Weights in the first convolution layer

# Detach and create a numpy copy, do some modifications on it
weight = weight.detach().cpu().numpy()
weight[0,0,0,:] = 0.0

# Now replace the whole weight tensor
net.layer1[0].weight = torch.nn.Parameter(torch.from_numpy(weight))
print(list(net.parameters()))

[Parameter containing:
tensor([[[[ 0.0000,  0.0000],
          [-0.0865,  0.1675]]]], requires_grad=True), Parameter containing:
tensor([[[[-1.4364e-01,  2.9724e-01],
          [ 3.0464e-04, -4.9807e-01]]]], requires_grad=True), Parameter containing:
tensor([[-0.0283,  0.0109, -0.0077,  ..., -0.0016, -0.0108, -0.0179]],
       requires_grad=True), Parameter containing:
tensor([-0.0004], requires_grad=True)]
user_16071040
Related