I have to convert large CZI microscopy images about +50GB in size into compress JP2 images for post analysis. The JP2 images need to be compress in order to save disk space and be post analyzed using software. My current setup only has 8 GB of ram available, so I need to be able to process this large images with my ram limited workstation.
I have managed to write scripts that convert smaller CZI images about 5 GB into JP2. I do this by reading a compress representation of the image into memory. However, when I try to do the same trick with the 50 GB images everything comes crashing down.
The follow is a representation of my workflow:
- Read the CZI image into memory and store it in a numpy array
- Save the numpy array into jp2 format using
glymur. To be able to write an image in JP2 format withglymurthe whole image needs to be loaded into memory. This is obviously a huge limitation when working with large images.
I would like to read a chunk of the CZI image and then write it into a JP2 image. This process should be repeated until the CZI image has been fully converted into its JP2 representation. If someone can show me how to write a JP2 image in chunks that would be enough to get the ball rolling, since I have seen documentation on reading chunks of CZI images into memory.
I appreciate any help or suggestions. Thank you in advance for your time.