Python: h5py gives OSError: Can't read data (inflate() failed) even though it's opened it before

user_7379587 0
Viewed 6473

Python 3.5. I have a few hundred .mat mat files (version 7.3) in a directory. I am looping through all of them to extract two different parts of data. I loop through and get and get the first lot with no problems at all but when I do the exact same thing again only I extract a different part of the data I get the following error:

Traceback (most recent call last):
  File "v73_test.py", line 43, in <module>
    mrfs_data = extract.convert1simProteinComCountsIntoDataFrame(path2mats)
  File "/home/oli/Downloads/PhD/wc/mg/version_73_stuff/functions_for_joshuas_matFiles/extract_matFile_data_v73.py", line 586, in convert1simProteinComCountsIntoDataFrame
    raw_data = getMatureProteinComplexs(path2mats, state_no)
  File "/home/oli/Downloads/PhD/wc/mg/version_73_stuff/functions_for_joshuas_matFiles/extract_matFile_data_v73.py", line 53, in getMatureProteinComplexs
    if len(np.array(state_file['ProteinComplex']['counts']).shape) == 3:
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/home/oli/virtualenvs/standard_python3.5/lib/python3.5/site-packages/h5py/_hl/dataset.py", line 696, in __array__
    self.read_direct(arr)
  File "/home/oli/virtualenvs/standard_python3.5/lib/python3.5/site-packages/h5py/_hl/dataset.py", line 657, in read_direct
    self.id.read(mspace, fspace, dest, dxpl=self._dxpl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5d.pyx", line 181, in h5py.h5d.DatasetID.read
  File "h5py/_proxy.pyx", line 130, in h5py._proxy.dset_rw
  File "h5py/_proxy.pyx", line 84, in h5py._proxy.H5PY_H5Dread
OSError: Can't read data (inflate() failed)

So the file is definitely there and accessible so the only thing I can think of is the data is corrupted but if that is the case surely I wouldn't have been able to extract any data from it all?

h5py python-3.x numpy python
2 Answers

Python 3.6. I experienced a similar problem, whereby I could read output data (for a given grid cell in a 2D SHETRAN hydrological model) e.g., up to the 24th time step:

 f['VARIABLES']['  6 srf_dep']['value'][0,0,24]

 Out[272]: -1.0

But by the 240th time step, I was getting the same error:

['VARIABLES']['  6 srf_dep']['value'][0,0,240]


 Traceback (most recent call last):

  File "<ipython-input-273-1f12011309c3>", line 1, in <module>
    f['VARIABLES']['  6 srf_dep']['value'][0,0,240]

  File "h5py\_objects.pyx", line 54, in h5py._objects.with_phil.wrapper

  File "h5py\_objects.pyx", line 55, in h5py._objects.with_phil.wrapper

  File "C:\Anaconda3\lib\site-packages\h5py\_hl\dataset.py", line 496, in __getitem__
    self.id.read(mspace, fspace, arr, mtype, dxpl=self._dxpl)

  File "h5py\_objects.pyx", line 54, in h5py._objects.with_phil.wrapper

  File "h5py\_objects.pyx", line 55, in h5py._objects.with_phil.wrapper

  File "h5py\h5d.pyx", line 181, in h5py.h5d.DatasetID.read

  File "h5py\_proxy.pyx", line 130, in h5py._proxy.dset_rw

  File "h5py\_proxy.pyx", line 84, in h5py._proxy.H5PY_H5Dread

 OSError: Can't read data (inflate() failed)

I solved this by deleting the problematic H5 file, reopening my IDE and running again. Thanks for the solution.

user_116771250
Related