What is the difference between the predict and predict_on_batch methods of a Keras model?

user_3860928 0
Viewed 33095

According to the keras documentation:

predict_on_batch(self, x)
Returns predictions for a single batch of samples.

However, there does not seem to be any difference with the standard predict method when called on a batch, whether it being with one or multiple elements.

model.predict_on_batch(np.zeros((n, d_in)))

is the same as 

model.predict(np.zeros((n, d_in)))

(a numpy.ndarray of shape (n, d_out)

keras deep-learning python
3 Answers

It seems predict_on_batch is a lot faster compared to predict if executed on a single batch.

  • batch & model information
    • batch shape: (1024, 333)
    • batch dtype: float32
    • model parameters: ~150k
  • timeit result:
    • predict: ~1.45 seconds
    • predict_on_batch: ~95.5 ms

In summary, predict method has extra operations to ensure a collection of batches are processed right, whereas, predict_on_batch is a lightweight alternative to predict that should be used on a single batch.

user_114784780
Related