Let's say I have a list of numbers stored as string and want to put it in a DataFrame while setting the dtype. This is on Python 3.9 with pandas 1.3.5.
import pandas as pd
pd.__version__
# '1.3.5'
test = ['15731245100']
df = pd.DataFrame(test, columns=["id"], dtype="float64")
# works
However, for integer types, this fails and it is beyond my understanding why. With pandas 1.1.5 on Python 3.6 I did not have this problem.
A. For "int64" conversion I get a ValueError:
df = pd.DataFrame(test, columns=["id"], dtype="int64")
Traceback (most recent call last):
File "E:\SOFTWARE\Python\Python39\lib\site-packages\IPython\core\interactiveshell.py", line 3441, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-4-3c90fbd765c6>", line 1, in <module>
df = pd.DataFrame(test, columns=["id"], dtype="int64")
File "E:\SOFTWARE\Python\Python39\lib\site-packages\pandas\core\frame.py", line 711, in __init__
mgr = ndarray_to_mgr(
File "E:\SOFTWARE\Python\Python39\lib\site-packages\pandas\core\internals\construction.py", line 313, in ndarray_to_mgr
values = sanitize_array(
File "E:\SOFTWARE\Python\Python39\lib\site-packages\pandas\core\construction.py", line 545, in sanitize_array
subarr = _try_cast(data, dtype, copy, raise_cast_failure)
File "E:\SOFTWARE\Python\Python39\lib\site-packages\pandas\core\construction.py", line 754, in _try_cast
subarr = maybe_cast_to_integer_array(arr, dtype)
File "E:\SOFTWARE\Python\Python39\lib\site-packages\pandas\core\dtypes\cast.py", line 2069, in maybe_cast_to_integer_array
raise ValueError("Trying to coerce float values to integers")
ValueError: Trying to coerce float values to integers
B. For "uint64" conversion I get a TypeError:
df = pd.DataFrame(test, columns=["id"], dtype="uint64")
Traceback (most recent call last):
File "E:\SOFTWARE\Python\Python39\lib\site-packages\IPython\core\interactiveshell.py", line 3441, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-5-65933034c7a7>", line 1, in <module>
df = pd.DataFrame(test, columns=["id"], dtype="uint64")
File "E:\SOFTWARE\Python\Python39\lib\site-packages\pandas\core\frame.py", line 711, in __init__
mgr = ndarray_to_mgr(
File "E:\SOFTWARE\Python\Python39\lib\site-packages\pandas\core\internals\construction.py", line 313, in ndarray_to_mgr
values = sanitize_array(
File "E:\SOFTWARE\Python\Python39\lib\site-packages\pandas\core\construction.py", line 545, in sanitize_array
subarr = _try_cast(data, dtype, copy, raise_cast_failure)
File "E:\SOFTWARE\Python\Python39\lib\site-packages\pandas\core\construction.py", line 754, in _try_cast
subarr = maybe_cast_to_integer_array(arr, dtype)
File "E:\SOFTWARE\Python\Python39\lib\site-packages\pandas\core\dtypes\cast.py", line 2059, in maybe_cast_to_integer_array
if is_unsigned_integer_dtype(dtype) and (arr < 0).any():
TypeError: '<' not supported between instances of 'str' and 'int'
What's happening here? Why does float conversion work but not the integer conversion? This is not affected by the size/length of the list nor the elements in it.