Is it possible to make Keras exploit as much as possible from each single GPU?
For example, with 2 GPUs of 10GB each and a 1-GB memory model I would expect a 20x increase training time by dividing the training set and cloning the model 10 times on each GPU.
Instead, I only found the multi_gpu_model option ( https://keras.io/utils/#multi_gpu_model ), which replicates the model on several GPUs. This helps in only 2x better performance, since it replicates the model once on each GPU.
Using fit_generator options such as use_multiprocessing=True does not help in this manner.
Is such a replication/performance increase possible?
Thanks.