RNN input shapes for timeseries with sliding window

Question

Hi I am following examples from the book Time Series Forecasting using Deep Learning: Combining PYTORCH, RNN, ... Unfortunately some of the function in the book are incomplete and hence I am struggling to figure out what the input shapes should be for the RNN Layer ...

# Sliding window function
def sliding_window(ts, features):

    x = []
    y = []

    for i in range(features + 1, len(ts) + 1):
        x.append(ts[i - (features + 1):i - 1])
        y.append([ts[i - 1]])

    return x,y

# Interpolates any missing values if any
def interpolated_timeseries(ts:pd.Series):
    return ts.interpolate()

# Returns timeseries sequence, sliding window size
def get_training_datasets(ts, features, test_size):

    ts = interpolated_timeseries(ts).tolist()
    X,y = sliding_window(ts, features)
    
    
    # Test split
    X_train, y_train, X_test, y_test = X[:-test_size],y[:-test_size],X[-test_size:],y[-test_size:]
    # Validation split
    val_size = int(0.35 * len(X_train))
   
    X_train,y_train,X_val,y_val = X_train[:-val_size],y_train[:-val_size],X_train[-val_size:],y_train[-val_size:]

    return X_train, y_train, X_val, y_val, X_test, y_test 

x_train,y_train,x_val,y_val,x_test,y_test = get_training_datasets(
    ts = pd.Series(scaled_ts.flatten()), 
    features = features, 
    test_size = 300
)

x_train = torch.FloatTensor(x_train)
...

The get_training_datasets() function is from a previous chapter as I couldn't find the function in the same chapter and some amendments had to be made work with the dataset. It return which returns a training dataset of size torch.Size([78477,240]) as well as testing and validation sets

# RNN Model
class RNN(nn.Module):
    def __init__(self, hidden_size, in_size, out_size):
        super(RNN, self).__init__()
        self.rnn = nn.RNN(input_size = in_size, hidden_size = hidden_size, batch_first = True)
        self.fc = nn.Linear(in_features = hidden_size, out_features = out_size)

    def forward(self, x, h_0 = None):
        rnn_out, h_n = self.rnn(x, h_0) 
        y_hat = self.fc(rnn_out.view(len(x), -1))
        return y_hat[-1], rnn_out[:,-1] # last hidden state

model = RNN(hidden_size = 24, in_size = 1, out_size = 1)

# Training
optimizer = optim.Adam(params = model.parameters(), lr = 0.02)
mse_loss = nn.MSELoss()
best_model = None
min_val_loss = sys.maxsize

training_loss = []
validation_loss = []

for t in range(training_epochs):
    prediction, _ = model(x_train.unsqueeze(0))
    ...

In the book they run training loop using, model.forward(x_train), however I figured I need add an extra dimension to account for the batch size (which I am going to guess is accounted for in the get_training_dataset() function in the book)

Shape of training set I input, torch.Size[1,78477,240] (x_train.unsqueeze(0).shape) which follows the format for batch=True from pytorch documentation (batch_size, sequence_length, input_size)

I ofcourse get a dimension error RuntimeError: input.size(-1) must be equal to input_size. Expected 1, got 240

So I am wondering what seems incorrect. Also I am a bit confused on why the book inputs all the training data at once when considering a sliding window. I may be wrong but I was under the impression that you add the data relevent to each sliding window and get the prediction step by step using inner for loop.

Something of the sorts of

for t in range(training_epochs):
    for x_sliding in x_train:
       prediction, _ = model(x_sliding.unsqueeze(0))

Any explanation on this is much appreciated Thanks

RNN input shapes for timeseries with sliding window

0 Answers