Pruning using Pytorch on a complicated model

user_13965184 0
Viewed 613

So I am trying to use torch.nn.utils.prune.global_unstructured.

I did it on a simple model and that worked. model.cov2 or other layers and that works. I am trying to do it on a model that's (nested)? I get errors as:

AttributeError: 'CNN' object has no attribute 'conv1'

and other errors. I tried everything to access this deep cov1, but I couldn't.

You can find the model code below:

class CNN(nn.Module):
def __init__(self):
    """CNN Builder."""
    super(CNN, self).__init__()

    self.conv_layer = nn.Sequential(

        # Conv Layer block 1
        nn.Conv2d(in_channels=3, out_channels=32, kernel_size=3, padding=1),
        nn.BatchNorm2d(32),
        nn.ReLU(inplace=True),
        nn.Conv2d(in_channels=32, out_channels=64, kernel_size=3, padding=1),
        nn.ReLU(inplace=True),
        nn.MaxPool2d(kernel_size=2, stride=2),

        # Conv Layer block 2
        nn.Conv2d(in_channels=64, out_channels=128, kernel_size=3, padding=1),
        nn.BatchNorm2d(128),
        nn.ReLU(inplace=True),
        nn.Conv2d(in_channels=128, out_channels=128, kernel_size=3, padding=1),
        nn.ReLU(inplace=True),
        nn.MaxPool2d(kernel_size=2, stride=2),
        nn.Dropout2d(p=0.05),

        # Conv Layer block 3
        nn.Conv2d(in_channels=128, out_channels=256, kernel_size=3, padding=1),
        nn.BatchNorm2d(256),
        nn.ReLU(inplace=True),
        nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, padding=1),
        nn.ReLU(inplace=True),
        nn.MaxPool2d(kernel_size=2, stride=2),
    )


    self.fc_layer = nn.Sequential(
        nn.Dropout(p=0.1),
        nn.Linear(4096, 1024),
        nn.ReLU(inplace=True),
        nn.Linear(1024, 512),
        nn.ReLU(inplace=True),
        nn.Dropout(p=0.1),
        nn.Linear(512, 100)
    )


def forward(self, x):
    """Perform forward."""
    # conv layers
    x = self.conv_layer(x)
    # flatten
    x = x.view(x.size(0), -1)
    # fc layer
    x = self.fc_layer(x)
    return x

How can I apply pruning on this model?

pruning pytorch google-colaboratory machine-learning python
2 Answers

Your modules are not names 'conv1' or 'conv2', you can see the names using the named_modules generator. From above, you have a 'conv_stem' which can be indexed as model.conv_stem[0] to access. You can iterate over modules to create a dict like:

parameters_to_prune = (
    (model.conv1, 'weight'),
    (model.conv2, 'weight'),
    (model.fc1, 'weight'),
    (model.fc2, 'weight'),
    (model.fc3, 'weight'), )

and pass this in. See for more: https://colab.research.google.com/github/pytorch/tutorials/blob/gh-pages/_downloads/f40ae04715cdb214ecba048c12f8dddf/pruning_tutorial.ipynb#scrollTo=UVFjM079F0Oi

user_93497850

Use this method to see the names of layers

for layer_name, param in model.named_parameters():
    print(f"layer name: {layer_name} has {param.shape}")

and pass those names to prune method

for eg , in prune.random_unstructured(module_name, name="weight", amount=0.3)

user_133325820
Related