Object detection model prediction times in node.js differ greatly from python

Viewed 134

I converted models (ssd_mobilenet_v1_ppn_coco, ssd_mobilenet_v2_coco, ssd_mobilenet_v1_coco) from the tensorflow model zoo using tfjs-converter. However, my prediction times in node.js (4 seconds per frame) in general seem to wildly differ from my prediction times when using the same models in python (0.1 seconds per frame). What is causing this disparity in prediction times? EDIT: I used the frozen_model.pb for Python, but converted the also-included saved_model into tfjs models.

For reference, I am using the models for real-time object detection, reading inputs from the webcam using opencv. I directly used the models without training, but I intend to train it on my own data once I resolve the performance issues.

I have also tested the coco_ssd implementation included in @tensorflow-models for node.js, which references the model found here: ssdlite_mobilenet_v2. Comparing the times between: ssd_mobilenet_v2_coco(3-4 seconds per frame) and this model(0.3-0.7 seconds per frame). I believe this suggests I can optimize the zoo models further?

This is the code I used to benchmark the models in Node.js:

async function load() {
    model = await tf.loadGraphModel('file://./javascript_models/ssdlite_mobilenet_v2_js/model.json')
    return model
}

const model = await load()

let t = Date.now() 
outputs = await model.executeAsync({image_tensor: input},
            ['detection_boxes:0', 
            'num_detections:0']
)
t = Date.now() - t
console.log('execute_time: ' + t)

This is the code I used to benchmark the code in python:

#Load graph
def load():
    detection_graph = tf.Graph()
    with detection_graph.as_default():
        od_graph_def = tf.GraphDef()
        with tf.gfile.GFile('./mobilenet_ppn/frozen_inference_graph.pb', 'rb') as fid:
            serialized_graph = fid.read()
            od_graph_def.ParseFromString(serialized_graph)
            tf.import_graph_def(od_graph_def, name='')
    return detection_graph

detection_graph = load()
with detection_graph.as_default():
        with tf.Session(graph=detection_graph) as sess:
            # Definite input and output Tensors for detection_graph
            image_tensor = detection_graph.get_tensor_by_name('image_tensor:0')

            # Each box represents a part of the image where a particular object was detected.
            detection_boxes = detection_graph.get_tensor_by_name(
                'detection_boxes:0')

            # Each score represent how level of confidence for each of the objects.
            # Score is shown on the result image, together with the class label.

            detection_scores = detection_graph.get_tensor_by_name(
                'detection_scores:0')
            detection_classes = detection_graph.get_tensor_by_name(
                'detection_classes:0')
            num_detections = detection_graph.get_tensor_by_name('num_detections:0')

t = time.time()
#the actual prediction
(boxes, scores, classes, num) = sess.run(
                    [detection_boxes, detection_scores,
                        detection_classes, num_detections],
                    feed_dict={image_tensor: frame_expanded})
#print the time taken to execute
print('execute time: ', time.time()- t)

I expected the time taken for the model prediction to be on the same magnitude, instead predictions on python run up to 100 times faster compared to node.js.

0 Answers
Related