I am training a CNN model with frames extracted from Drive&Act videos (n°=19). In order to avoid that RAM goes out of memory, I fit the model (a typical CNN) with only one video (which contains all the classes). However I can't do more than that because RAM goes out of memory.
- The resolution of the video is (424, 512)@30fps.
- The frames extracted through openCV2 have a shape of (424,512,3). Then I resized them to (256,256, 3, dtype=uint8) . The size of each frame is 256x256x3/1000000 ~ 0.196 MB
- The video duration is approximately 20 minutes and their size is less than 100MB
- So the number of frames is 20x60x30 = 36000 frames.
- Multiplying that for the single frame dimension results in ~ 7GB (for a single sequence of frames)
My questions are:
- Is there some way to reduce the size of frames sequence?
- What is the most efficient way to process this great amount of data with a mini-batch technique?
I already used a Python generator fed into the model.fit method of Keras model to reduce RAM usage. However the main problem is that the first batches has the same class, thus the model start to overfitting that kind of image. One way can be perform a first training step with a bigger subset of data and then apply a mini-batch training method?