I am using pytorch to build an LSTM. The goal:
- Input a series of tensors and get a prediction for each tensor
My data:
- 100 videos. Each video is 100 seconds long has been split into 1 seconds chunks (at 24 fps). Each frame within each chunk is a
(224,224,3)matrix. Total input data looks like this:
(100, 100, 24, 224, 224, 3)
- 100 videos
- 100 seconds per video
- 24 frames per chunk
- (224,224,3) tensor per frame
Each video measures the same thing so I can treat them all as input data. However, as they represent different trials, the sequential nature that occurs within each video does not continue across all videos. As a result, I don't want to join all of my data into 1 training set.
I believe I want to batch my data according to each video, such as 100 batches for 100 videos. Is this the right approach? I want the LSTM to remember sequential information within each video, but not across videos.
Will writing a custom DataLoader function with pytorch be the right thing to do, or is my understanding inadequate?