How to load 2D data into an LSTM in pytorch

user_10323342 0
Viewed 3422

I have a series of sine waves that i have loaded in using a custom dataloader. The data is converted to a torch tensor using from_numpy. I then try to load the data using an enumerator over the train_loader. The iterator is shown below.

for epoch in range(epochs):
    for i, data in enumerate(train_loader):
        input = np.array(data)
        train(epoch)

The error i receive is:

RuntimeError: input must have 3 dimensions, got 2

I know i need to have my input data in [sequence length, batch_size, input_size] for an LSTM but i have no idea how to format my array data of 1000 sine waves of length 10000.

Below is my training method.

def train(epoch):
    model.train()
    train_loss = 0
    def closure():
        optimizer.zero_grad()
        print(input.shape)
        output = model(Variable(input))
        loss = loss_function(output)
        print('epoch: ', epoch.item(),'loss:', loss.item())
        loss.backward()
        return loss
    optimizer.step(closure)

I thought i would try add (seq_length, batch_size, input_size) in a tuple but this cant be fed into the network. Further to this my assumption was that the dataloader fed batch size into the system. Any help would be appreciated.

edit:

Here is my sample data:

T = 20
L = 1000
N = 100

x = np.empty((N, L), 'int64')
x[:] = np.array(range(L)) + np.random.randint(-4 * T, 4 * T, N).reshape(N, 1)
data = np.sin(x / 1.0 / T).astype('float64')
torch.save(data, open('traindata.pt', 'wb'))
rnn lstm pytorch python
1 Answers

Can you share a simple example of your data just to confirm?

Also, you have to have a different order for your shape. Generally, the first dimension is always batch_size, and then afterwards the other dimensions, like [batch_size, sequence_length, input_dim].

One way to achieve this, if you have a batch size of 1, is to use torch.unsqueeze(). This allows you to create a "fake" dimension:

import torch as t
x = t.Tensor([1,2,3])
print(x.shape)
x = x.unsqueeze(dim=0) # adds a 0-th dimension of size 1
print(x.shape)
user_36072030
Related