I am trying to write out a 3d array to a zarr store. Should be straightforward but the to_zarr step is failing with an error message indicating it is trying to write out a 4d array and running out of memory. The 3d array is obtained from a 4d array using isel and slicing out the values for a single value along the fourth dimension, such that the result is a 3d array. See the attached image showing the field I am trying to write out after the isel operation.
I then ran the following commands:
ds_sst = sst.chunk('auto').to_dataset(name='sst')
ds_sst.to_zarr('./test.zarr',mode='w')
Here is the error message I am getting:
MemoryError: Unable to allocate 354. GiB for an array with shape (5928, 50, 489, 655) and data type float32
The weird part is the fourth dimension which should have been squeezed out in the original array after the isel operation has a length of 50. If you look at the error message, it suggests the isel operation did nothing? Yet the image shows the field is indeed 3d with a size ~11 Gb. So I am puzzled why the to_zarr operation needs 354 Gb of memory.