PyTorch 1D CNN Problems

user_16652188 0
Viewed 295

I am having quite a lot of trouble implementing a 1D CNN in PyTorch. The idea is we are using xTrainVar to predict yTrainVar, and xTestVar to predict yTestVar. Time series.

This line:

pred = prod_outputs(train_loader, model)

Creates a problem here:

_, predictions = torch.max(scores, 1)

That results in this error:

IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

Normally I would not dump a complete codebase, but I just cannot see my error so here it is:

#################################
# Load the libraries
#################################
import torch
import torchvision
import torch.nn.functional as F
import torchvision.datasets as datasets
import torchvision.transforms as transforms
from torch import optim
from torch import nn
from torch.utils.data import DataLoader 
from tqdm import tqdm 

from sklearn.metrics import r2_score, mean_absolute_error, mean_squared_error
import numpy as np
import torch.utils.data as data_utils


#################################
# Prepare the data
#################################
xTrainVar = [[8.1,4.1,4.1,4.3], [3.9, 3.8, 3.9, 3.7], [3.8, 3.7, 3.8, 3.8], [3.9, 3.8, 4.1, 4.4]]
yTrainVar = [3.9, 3.8, 3.9, 4.4]
xTestVar = [[8.1,4.1,4.1,4.3], [3.9, 3.8, 3.9, 3.7], [3.8, 3.7, 3.8, 3.8], [3.9, 3.8, 4.1, 4.4],[3.9, 3.8, 4.1, 4.4]]
yTestVar = [3.9, 3.8, 3.9, 4.4, 4.4]

# Convert to Tensors
# Prepare training data
train_data = torch.tensor(np.array(xTrainVar, dtype=np.float32))
train_target = torch.tensor(np.array(yTrainVar, dtype=np.float32))

train_data = train_data.unsqueeze(1)  # try to get the right shape [4,1,4]

# Reshape the data to be in line with the other shape
new_shape = (len(yTrainVar), 1, 1) # There used to be another ,1 here
train_target = train_target.view(new_shape)
                                     
train_tensor = data_utils.TensorDataset(train_data, train_target)
train_loader = data_utils.DataLoader(dataset=train_tensor, batch_size=32,shuffle=False)

# Prepare test data
test_data = torch.tensor(np.array(xTestVar, dtype=np.float32))
test_target = torch.tensor(np.array(yTestVar, dtype=np.float32))

test_data = test_data.unsqueeze(1) # try to get the right shape [5,1,4]

# Reshape the data to be in line with the other shape
new_shape = (len(yTestVar), 1,1)
test_target = test_target.view(new_shape)

test_tensor = data_utils.TensorDataset(test_data, test_target)
test_loader = data_utils.DataLoader(dataset=test_tensor, batch_size=32)

class NN(nn.Module):
    def __init__(self, input_size, num_classes):
        super(NN, self).__init__()
        self.conv1d = nn.Conv1d(in_channels=1, out_channels=4, kernel_size=4, stride=1, padding=0)
        self.relu = nn.ReLU(inplace=True)
        self.fc1 = nn.Linear(16,50)
        self.fc2 = nn.Linear(50,1).

    def forward(self, x): 
        x = self.conv1d(x)
        x = self.relu(x)
        x = x.view(-1)
        x = self.fc1(x)
        x = self.relu(x)
        x = self.fc2(x)
        return x


# Set the hyperparameters
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
input_size = 1*4
num_classes = 1
learning_rate = 0.001
batch_size = 64
num_epochs = 1

model = NN(input_size=input_size, num_classes=num_classes).to(device)

criterion = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)


#################################
# Train model using training data
#################################
for epoch in range(num_epochs):
    # Fit the model to the training data
    for batch_idx, (data, targets) in enumerate(tqdm(train_loader)):
        # Get data to cuda if possible
        data = data.to(device=device)
        targets = targets.to(device=device)
        
        # forward
        scores = model(data)
        loss = criterion(scores, targets)

        # backward
        optimizer.zero_grad()
        loss.backward()

        # gradient descent or adam step
        optimizer.step()


def prod_outputs(loader, model):
    model.eval() # Place the model into evaluation mode.

    with torch.no_grad():
        for x, y in loader:
            x = x.to(device=device)
            y = y.to(device=device)
            scores = model(x)
            _, predictions = scores.max(1)
            print(predictions)
    model.train() # Return the model to training mode once we are done.
    return scores # This used to be predictions


#################################
# Evaluate the model
#################################
# This is where I am up to with the work.
pred = prod_outputs(train_loader, model) # Use train dependent variable (x) to generate predictions of independent variable (y)
pred = pred.numpy()
conv-neural-network pytorch python
1 Answers

You need to take a step back and assess the situation:

  • You have an input shaped (bs, c, w) as (4, 1, 4). So a batch size of 4 and a length of 4 with a single channel. Going through the convolutional layer, this becomes (bs, 4, 1) since the kernel size is (4,), there is essentially a single value per channel.

  • Right after you are applying a reshape which flattens the whole tensor, this is generally a bad idea. Since you are actually flattening the batch elements together. Instead, you should be looking to keep them separated. You can use nn.Flatten for that. For instance, set self.flatten = nn.Flatten() in __init__ then use x = self.flatten(x) instead of x = x.view(-1) in the forward definition.

  • This being said, flattening except for the first axis (the batch axis) will result in a tensor of shape (4, 4). So, ultimately your first dense layer should have the corresponding number of neurons, i.e. 4. Something like self.fc1 = nn.Linear(4, 50).

Don't take this for granted, I'm explaining what is wrong in your code, not giving you an architecture to stick with. The modications I'm providing you with will run, no guarantee it will train... This is a problem for another day!

user_63313690
Related