Cast string into Pandas or Numpy dtype using same logic as read_csv?

user_3620725 0
Viewed 69

How can I parse strings using the same logic Pandas would use when reading a CSV, where casting "False" to bool would give me False. I have text values entered by users that I need to insert into a DataFrame, they should automatically be cast to the dtype of the column being inserted to using this logic. The example below shows an attempt to insert a value into a boolean column but the result is wrong.

import pandas as pd

x = pd.DataFrame([{'id': 0, 'flag': True},
                  {'id': 1, 'flag': False},
                  {'id': 2, 'flag': True}])

text = "False"
value = x['flag'].dtype.type(text)  # Want this to be False not True
x.loc[0, 'flag'] = value
pandas numpy python
3 Answers

Use json.loads() and then convert the dtype of flag to its previous type. It will work for "False", "false", "1", "0" etc

previous_type = x.flag.dtype
x.loc[0, 'flag'] = json.loads(text.lower())
x.flag = x.flag.astype(previous_type)

Complete code:

import pandas as pd
import json

x = pd.DataFrame([{'id': 0, 'flag': True},
                  {'id': 1, 'flag': False},
                  {'id': 2, 'flag': True}])
text = "False"
previous_type = x.flag.dtype
x.loc[0, 'flag'] = json.loads(text.lower())
x.flag = x.flag.astype(previous_type)
print(x)

    id  flag
0   0   False
1   1   False
2   2   True
user_135770630

Please remember, In Python

>>bool("faa")
True
>>bool("True")
True
>>bool("False")
True
>>bool("")
False

So In your case,

import pandas as pd

x = pd.DataFrame([{'id': 0, 'flag': True},
                  {'id': 1, 'flag': False},
                  {'id': 2, 'flag': True}])

text = bool("")
value = x['flag'].dtype.type(text)  # Want this to be False not True
print(value) // False
x.loc[0, 'flag'] = value

should do

Another solution might be

import pandas as pd

x = pd.DataFrame([{'id': 0, 'flag': True},
                  {'id': 1, 'flag': False},
                  {'id': 2, 'flag': True}])

text = "False"
value = x['flag'].dtype.type(eval(text))  # Want this to be False not True
print(value) // False
x.loc[0, 'flag'] = value
user_123740770

Here is a workaround that works but may not have very good performance

from io import StringIO
import pandas as pd
value = pd.read_csv(StringIO(text), dtype=column_dtype, header=None).values[0][0]
user_36207250
Related