Matplotlib savefig() slow ...just the way things are or ideas to speed up?

user_9423009 0
Viewed 692

I've got some code where I'm using MPL (not pyplot) via imshow() to show some arrays and then am using savefig() to save them as PNG files.

The arrays are approx 3,000 x 4,000 in size.

My problem is that saving is taking a long time - on the order of 4 seconds or so per image.

Minor Details

  • The arrays are floats
  • I'm using cmap of gray
  • I'm making sure the figure resolution is the same as the images, and the axes fills the entire figure (so fig size * dpi matches exactly the shape of the arrays)
  • I'm using imshow() with interpolation of none.
  • Running on macbook pro - but running on anything else is about the same (assuming SSD)

The slowness seems to be due to CPU bottleneck. Using time wrapped around my code shows real and user time to be about the same, so it doesn't seem to be a IO bottleneck.

However, (very curiously!), if I run the code via Multiprocessing in multiple processes, it doesn't seem to help much with overall real time (even with 4 cores).

Questions

  • Is saving to PNG taking around 4 seconds 'normal'?
  • Any tips or ideas on how to speed things up?
matplotlib python-3.x
2 Answers

Never tried it but I think you could try to run the code via Multiprocessing using the GPU (which may be more suited for the process) if you have Nvidia graphic card. https://documen.tician.de/pycuda/ Other than that I don't think you can speed up the process more.

user_142150780

From the details of what you are doing it sounds like you just want to save the array as a (false) color image. You are very carefully setting up Matplotlib to do that for you, but we do not have the logic to notice we can take any short-cuts so are still going through all of the resampling logic.

You can generate an equivalent output more simply via:

import matplotlib.colors as mcolors
import matplotlib.cm as mcm
import numpy as np
import PIL
import time


# "data"
my_data = np.random.randn(3000, 4000) * 50

start_time = time.monotonic()
# to scale the data to [0, 1]
my_norm = mcolors.Normalize(-50, 50)
# to map the scaled data to gray scale RGB
my_cmap = mcm.get_cmap('gray')
setup_time = time.monotonic()
# apply the above transforms
color_mapped = my_cmap(my_norm(my_data), bytes=True)
mapping_time = time.monotonic()
# use pillow to save the png
PIL.Image.fromarray(color_mapped).save('/tmp/so.png', compress_level=1)
end_time = time.monotonic()


print(f"saving took {end_time - start_time}")
print(f"  setup took {setup_time - start_time}")
print(f"  mapping took {mapping_time - setup_time}")
print(f"  saving took {end_time - mapping_time}")

but a majority of the time is still spent in .save(...). By playing with compress_level you can make that line more/less expensive which suggests that cost is inside of libpng while compressing the data (see https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html#png).

user_3802310
Related