PyTorch: How to check if some weights are not changed during training?

There can be two ways around this:

First

        for name, param in model.named_parameters():
            if 'weight' in name:
                temp = torch.zeros(param.grad.shape)
                temp[param.grad != 0] += 1
                count_dict[name] += temp

This step comes in after your loss.backward() step in the training module. The count_dict[name] dictionary keeps track of the gradient updates. You can initialize it this way before the start of training:

    for name, param in model.named_parameters():
        if 'weight' in name:
            count_dict[name] = torch.zeros(param.grad.shape)

Now one more way would be to register a hook function and then create the hook function where you can even update the or modify the gradients if you want to. This is not necessary to keep track of the weight updates but then if you want to do something with the gradient, it comes in handy. Suppose, here I am randomly sparsifying the gradients.

def hook_fn(grad):
    '''
    Randomly sparsify the gradients
    :param grad: Input gradient of the layer
    :return: grad_clone - the sparsified FC layer gradients
    '''
    grad_clone = grad.clone()
    temp = torch.cuda.FloatTensor(grad_clone.shape).uniform_()
    grad_clone[temp < 0.8] = 0
    return grad_clone

And here I give the model the hook.

for name, param in model.named_parameters():
    if 'weight' in name:
            param.register_hook(hook_fn)

So, this might just sparsify the gradients for you, and you can keep track of gradients in the hook function itself in this way:

def hook_func(module, input, output):
    temp = torch.zeros(output.shape)
    temp[output != 0] += 1
    count_dict[module] += temp

Although, I won't recommend doing this. This is generally useful in the case of visualizing the forward pass features/activations. And also, input and output can confuse because the gradient and param inputs and outputs are reversed.