I am building an iPhone app with a vision processing component that could use image segmantation, especially if it can run at 30fps. I am starting the exploration process with this repo, which uses the 8bit model from Apple. The inference times I get on an iPhone 7 are on the order of 200ms on 513x513 images.
I am used to image models scaling linearly with the number of pixels, so I assumed if I changed the size of the input image to 320x240 (the size we eventually expect to use) then I would get a speed up factor of 513x513/(320x240), for inference times around 60ms. However, the inference time is still on the order of 200ms.
Further detail: I created the 320x240 model by starting with the Deep Lab mobilenetv2_coco_voc_trainaug from the Deep Lab Model Zoo and then used coremltools to convert it to a CoreML model with input size 320x240 and quantized to 8 bits using kmeans.
Does anyone have any insight into why I am not seeing a speedup? Is there some way to achieve the throughput I was expecting?