Fast image sequences / animation in Jupyter Notebook with matplotlib

I can't seem to find a simple and fast way of plotting image sequences with plain matplotlib in a Jupyter Notebook. I've tried FuncAnimation, fig.canvas.draw(), blitting, as well as just the standard imshow-pause combo; without success or with very slow refresh rate. I don't need the images to be interactive - they just need to be shown sequentially and can't pop up a new figure window for each image. I've seen many solutions here, with none seeming to work the way I want.

My general pipeline does significant processing, with each image generated and plotted within a while or for loop. FuncAnimation is not desirable since it requires passing a function handle and my use case involves many arguments and state variables that make it difficult to use.

The best I've got is the working example below using fig.canvas.draw() - showing that drawing time increases linearly per iteration, where I need it to remain constant!

import numpy as np
import matplotlib.pyplot as plt
from timeit import default_timer as timer
%matplotlib notebook

num_iters = 50
im = np.arange(60).reshape((15,4))

fig, ax = plt.subplots(1,1)
fig.show()
fig.canvas.draw()

iter_times = np.zeros(num_iters)
for i in range(num_iters):
    im = np.roll( a=im, shift=1, axis=0 )

    t0 = timer()
    ax.imshow(im.T, vmin=im.min(), vmax=im.max())
    ax.set_title('Iter # {}/{}'.format(i+1, num_iters))
    fig.canvas.draw()

    iter_times[i] = timer()-t0

plt.figure(figsize=(6,3))
plt.plot(np.arange(num_iters)+1, iter_times)
plt.title('Imshow/drawing time per iteration')
plt.xlabel('Iteration number')
plt.ylabel('Time (seconds)')
plt.tight_layout()
plt.show()