How to force data types with pandas DataFrame builder's function from_records?

user_5780022 0
Viewed 1016

When I build a pandas DataFrame from a list of float32 ndarrays, I get a pandas DataFrame with float64 items.

How can I get float32 items in the DataFrame ? 

       import numpy as np
       import pandas as pd

       # Create 2 dummy arrays instead of reading a bunch of float32 records in binary file
       record1 = np.array([202.1, 0.0], dtype='float32')
       print('record1 1st item type is', type(record1[0]))
       record2 = np.array([202.2, 0.0], dtype='float32')

       # Group records in list and create a dataframe from list
       records_list = [record1, record2]
       print('records_list 1st item of 1st item is', type(records_list[0][0]))

       # During dataframe construction, float32 items are converted to float64 items !?!
       df = pd.DataFrame.from_records(data=records_list)
       print('dataframe\'s types :')
       print(df.dtypes)

       # Real values are now different
       print('record1 1st item\'s value before then after:',  record1[0], df.iloc[0, 0])
       print('record2 1st item\'s value before then after:',  record2[0], df.iloc[1, 0])

       # Outputs
       # >>> record1 1st item type is <class 'numpy.float32'>
       # >>> records_list 1st item of 1st item is <class 'numpy.float32'>
       # >>> dataframe's types :
       # >>> 0    float64
       # >>> 1    float64
       # >>> dtype: object
       # >>> record1 1st item's value before then after: 202.1 202.10000610351562
       # >>> record2 1st item's value before then after: 202.2 202.1999969482422
pandas dataframe floating-point python
1 Answers

You can coerce a DataFrame to a specific dtype with astype:

df = pd.DataFrame.from_records(data=records_list).astype(np.float32)

But this will first build a DataFrame of float64 and then another one of float32

You can also directly specify the dtype at creation time if you use the constructor:

df = pd.DataFrame(data = records_list, dtype=np.float32)

This will directly build a float32 type dataframe.

user_35452730
Related