Pandas calling apply on an empty dataframe using the reduce option changes datatypes

user_1341773 0
Viewed 836

I understand that apply method is called even for the empty dataframe. When there is error inside the apply method it doesn't get propagated. I was looking at this stackoverflow link which suggests to use the reduce option so that the apply function is not called. Pandas: why does DataFrame.apply(f, axis=1) call f when the DataFrame is empty?

Consider this example, in Col1, everything is less than 10. So the dataframe is empty. when I use the reduce option, the datatype of col2 is changed. It converts the numbers to decimals.

    d = {'col1': [1, 2], 'col2': [3, 4]}
    df = pd.DataFrame(data=d)
    mask = df["col1"] > 10
    df.loc[mask, "col2"] = df[mask].apply(lambda x: x+2, axis=1, result_type='reduce')
    print(df)

Expected output

      col1    col2
   0    1       3
   1    2       4

Actual output:

      col1    col2
   0    1      3.0
   1    2      4.0

I am not sure why it converts the integers to decimals. Does anyone know how to avoid this?

pandas dataframe python
1 Answers

You can use pd.to_numeric() for downcasting to integer.

I will update this if I find something better to do this.

>>> import pandas as pd
>>> d = {'col1': [1, 2], 'col2': [3, 4]}
>>> df = pd.DataFrame(data=d)
>>> mask = df["col1"] > 10
>>> df.loc[mask, "col2"] = df[mask].apply(lambda x: x+2, axis=1, result_type='reduce')
>>>
>>> df
   col1  col2
0     1   3.0
1     2   4.0
>>>
>>> pd.to_numeric(df.col2, downcast='integer')
0    3
1    4
Name: col2, dtype: int8
>>>
>>> df.col2 = pd.to_numeric(df.col2, downcast='integer')
>>> df
   col1  col2
0     1     3
1     2     4
>>>
user_66151630
Related