decimals issue with pd.read_csv

user_8569166 0
Viewed 2018

I have Anaconda 3 on windows 10. I am reading csv files with an European format ( ',' as a separator). I use 'decimal' to specify the comma, but it does not apply to all the columns.

The csv file rows are like this :

2;PAU;11:21:19;00:00;00:00;0;0;2,102;0,00;20,75;20,75;0,00;0,00;0,00;0,00;0,00;0,00;0,00;0,00


x=pd.read_csv(file,sep=';',decimal=',',encoding='latin-1',low_memory=False)

the output is like this :

0 2 PAU 11:21:19 00:00 00:00 0 0 2,102 0.00 20.75 20.75 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

The european format was successfully taken into account for 20.75 but not for 2,102.

Any suggestion?

edit : still no clean solution, i managed by doing this :

x['Tension'] = x['Tension'].str.replace(',','.')
pandas csv python
2 Answers

It happens because your data is interpreted as a header. Read it with mentioning that you have no header:

x=pd.read_csv(file,sep=';',decimal=',',
    encoding='latin-1',low_memory=False,header=None)
user_64014030

Everyone coming from a google search like: pandas read_csv decimal not working, might have strings and floats in the same column. In this case pandas cannot apply the decimal option.

Have a look into the na_values parameter. I had an odd string that denoted NAN values. na_values let's you explicitly handle these.

user_125188620
Related