Initially, I was trying transfer learning in image classification model using 268 test images + 65 validation images of 256x256 sizes.
(The code I am using is from https://github.com/conan7882/GoogLeNet-Inception)
However even with Tesla K80(16 gb memory), I still can't train more than batch size of 5.
Now that I increased the size of dataset to 480 test images + 120 validation images, the out of memory error pops up again. I have to reduce the batch size to 2 to continue training.
Why does dataset size matter in memory usage? Shouldn't only batch size decides how much data is being processed by the GPU?