Python execution hangs when attempting to add a pytorch module to a sequential module in loop

user_7254514 0
Viewed 171

I am designing a neural network that has a sequential module that is composed of a variable number of linear layers depending on the initial size of the feature space. The problem is that on the first module that I append to my sequential module, the code stops executing and my runtime is crashed. Here is my code:

self.sequential = nn.Sequential()
input_dim = feature_size
output_dim = int(feature_size // scaling_factor)
while (output_dim > 1000):
    print("%s_%s" % (input_dim, output_dim))
    # Hangs on next line
    self.sequential.add_module("%s_%s" % (input_dim, output_dim), nn.Linear(input_dim, output_dim))
    input_dim = output_dim
    output_dim = int(input_dim // get_scaling_factor(input_dim))

What is the appropriate way to create a sequential module with a variable number of layers?

EDIT:

Turns out the code above is OK. The reason for which I was getting the hanging was that I was attempting to create a layer with way too many inputs and outputs for the interpreter to handle. (Both input_dim and output_dim were on the order of 100,000s).

sequential pytorch python
1 Answers

As mentioned in this PyTorch Forums thread, you can create a list of nn.Modules and feed them to an nn.Sequential constructor.

For example:

import torch.nn as nn

modules = []
modules.append(nn.Linear(10, 10))
modules.append(nn.Linear(10, 10))

sequential = nn.Sequential(*modules)

Also, as mentioned in PyTorch Documentations, you may create a sequential with names for each layer using a ordered Dictionary, created from a list of tuples; each containing the name of the layer and the layer itself. i.e.

model = nn.Sequential(OrderedDict([
          ('conv1', nn.Conv2d(1,20,5)),
          ('relu1', nn.ReLU()),
          ('conv2', nn.Conv2d(20,64,5)),
          ('relu2', nn.ReLU())
        ]))

More specifically, adapting your example to use this method:

from collections import OrderedDict

...

sequential_list = []
input_dim = feature_size
output_dim = int(feature_size // scaling_factor)
while (output_dim > 1000):
    print("%s_%s" % (input_dim, output_dim))
    name = "%s_%s" % (input_dim, output_dim)
    layer = nn.Linear(input_dim, output_dim)
    sequential_list.append((name, layer))
    input_dim = output_dim
    output_dim = int(input_dim // get_scaling_factor(input_dim))
self.sequential = nn.Sequential(OrderedDict(sequential_list))
user_153276140
Related