I am new to tensorflow and have not touch on multiprocessing. I am using a customized object detection model to process my images. The model is based on faster_rcnn_inception_v2_pets and i am able to use both CPU and GPU to process the images. Currently each image take about 4 secs to process. When i check the task manager, the CPU and GPU is not fully utilized. I found that for each image, at the start it uses the CPU like 95% of the time and when it goes to the GPU, the results are out almost instantly. My question is, is it possible to speed this up by using multi-core processing?
I have been reading up on multiprocessing and was wondering if it is possible to use multiprocessing in the CPU (example 8 core) to process images in parallel and queue them into the GPU. (I think i read somewhere that the GPU is only able to process 1 at a time.) Since the CPU is taking a long time to process 1 image while the GPU is very fast, if the above is possible it will cut down on processing time for a large number of images.
Thank you.