Convert a Pandas dataframe with a date column to a Vaex dataframe

user_8425663 0
Viewed 2830

I am trying to do the following

  1. load some data with string columns 
measurement_df = pd.read_csv('data/tag_measurements/all_measurements.csv')
measurement_df.head(3)
measurement_df
>> prints
.  timestamp               tag_1      tag_2        tag_3    
0   2018-01-01 11:09:00 0.729193    -0.236627   -1.968651   
1   2018-01-02 05:56:00 -2.812988   0.394632    -1.151147   
2   2018-01-03 00:37:00 0.363185    -0.089076   -1.509133

at this point the timestamp column is of type str: 

type(measurement_df.iloc[0]['timestamp'])
>> prints
str
  1. convert it to Vaex
vdf = vx.from_pandas(measurement_df)
vdf.head(3)
>> prints
#           tag_1          tag_2                  tag_3           index
0   0.7291933972260769  -0.2366268009370677  -1.9686509728501898    0
1   -2.8129876800434737 0.3946317890604529   -1.1511473058592252    1
2   0.3631852302577519  -0.08907562484360453 -1.5091330993605443    2

somehow I lose the timestamp column. Any ideas what could be going wrong?

vaex pandas dataframe timestamp python
1 Answers

If you would like to preserve the date/time format, especially while reading CSVs, i suggest you do :

df = pd.read_csv('myfile.csv', parse_dates=['datetime_col_1', 'datetime_col_2'])

you can also do:

df = vaex.read_csv('myfile.csv', parse_dates=['datetime_col_1', 'datetime_col_2'])

it is the same since it is using the pandas method in the background.

user_29965760
Related