pandas to_numeric couldn't convert string values to integers

user_766708 0
Viewed 2554

I am trying to use pandas.to_numeric to convert a series to ints. 

df['numeric_col'] = pd.to_numeric(df['numeric_col'], errors='raise')

I got errors,

Traceback (most recent call last):
  File "/home/user_name/script.py", line 86, in execute
data = module(**module_args).execute(data)
  File "/home/user_name/script.py", line 62, in execute
invoices['numeric_invoice_no'] = pd.to_numeric(invoices['numeric_invoice_no'], errors='raise')
  File "/usr/local/lib/python3.5/dist-packages/pandas/core/tools/numeric.py", line 126, in to_numeric
coerce_numeric=coerce_numeric)
  File "pandas/_libs/src/inference.pyx", line 1052, in pandas._libs.lib.maybe_convert_numeric (pandas/_libs/lib.c:56638)
ValueError: Integer out of range. at position 106759

if I change it to,

df['numeric_col'] = pd.to_numeric(df['numeric_col'], errors='coerce')

the values in numeric_col will not convert to ints, i.e. they are still strings.

if I changed to,

df['numeric_col'] = df['numeric_col'].astype(int)

I got error,

OverflowError: Python int too large to convert to C long

so I have to change it to,

df['numeric_col'] = df['numeric_col'].astype(float)

then there was no error generated.

The size of the series is about 994572, the strings in the column are like 52333612273, 56032860 or 02031757.

I am wondering what are the issues with to_numeric and astype here.

I am running Python 3.5 on Linux mint 18.1 64-bit.

pandas series dataframe python-3.x numpy
1 Answers

Maybe you have a comma(,) within your numeric string values or still having a null value(NaN) within the columns of your dataframe , so try to replace the commas with empty space using the .replace() method and then drop or fill in the Null values with .fillna() or .replace or .dropna()

before using df['DataFrame Column'] = df['DataFrame Column'].astype(int)

user_155985510
Related