I want to read only specific columns from HDF5 file and pass conditions on those columns. My concern is that I dont want to fetch all HDF5 file as dataframe in the memory. I want to get only my necessary columns with their conditions.
columns=['col1', 'col2']
condition= "col2==1"
groupname='\path\to\group'
Hdf5File=os.path.join('path\to\hdf5.h5')
with pd.HDFStore(Hdf5File, mode='r', format='table') as store:
if groupname in store:
df=pd.read_hdf(store, key=groupname, columns=columns, where=["col2==1"])
I get an error :
TypeError: cannot pass a column specification when reading a Fixed format store. this store must be selected in its entirety
Then I use below line which returns only specific columns:
df=store[groupname][columns]
But I dont know how can I pass condition on it.