I have a series of *.tiff movies that are too large for numpy to deal with (shape = (1, 9000, 512, 512)) and it seems like dask.array.image.imread could handle this (according to the answer of this question: out of core 4D image tif storage as hdf5 python).
When creating the array, dask.array.image.imread gives a MemoryError when trying to import any of the large files. It does, however, work on smaller stacks of files with shape = (1, 20, 512, 512), so I think it must be something about chunk sizes. I've tried to see if I can change the chunk size upon import within imread but I haven't found anything.
When dask's own imread didn't work, I tried the dask_imread package, which seemed to have the ability to split frames into chunks. Interestingly, it gives me a RuntimeWarning when creating the array, but upon print(foo_array) still returns
dask.array<from-value, shape=(1, 9000, 512, 512), dtype=uint16, chunksize=(1, 9000, 512, 512)>
With a chunksize of the entire file.
Trying .rechunk((1,20,512,512)) returns
dask.array<rechunk-merge, shape=(1, 9000, 512, 512), dtype=uint16, chunksize=(1, 20, 512, 512)>
as expected.
However, when running .compute() on any element of the array, for example print(foo_array[0,1234,123,123].compute()) gives a MemoryError.
I've tried importing as foo_array = imread(file_to_use,chunksize=(1,20,512,512)) but it seems that neitherimread takes chunksize as a kwarg.
How do I change the chunksize upon import in imread to for example (1,20,512,512) (which seems to be manageable)?