I am doing video classification for action detection using Keras (v.2.3.1). My model is CNN and LSTM. My dataset consists of videos of 3-7 seconds, each representing specific action. I use OpenCv to get frames from each video. But since video lengths are different, I get different number of frames for each video. However, I need to have the same number of frames for the LSTM layer. I searched a little bit and looks like padding along with a masking layer should do it, but I can’t figure out how to do this with Keras. Any help is appreciated. Here is my model:
conv_base = VGG16(weights= 'imagenet', include_top=False)
model = models.Sequential()
model.add(TimeDistributed(conv_base, input_shape = (n_timesteps, img_length, img_height, channel)))
model.add(TimeDistributed((Flatten())))
model.add(LSTM(units = lstm_cells))
model.add(Dropout(0.5))
model.add(Dense(512, activation='relu'))
model.add(Dense(256, activation='relu'))
model.add(Dense(100, activation='relu'))
model.add(Dense(1, activation='sigmoid'))