Applying a function to chunks of the Dataframe

user_14126173 0
Viewed 177

I have a Dataframe (df) (for instance - simplified version)

              A    B 
 0           2.0   3.0
 1           3.0   4.0

and generated 20 bootstrap resamples that are all now in the same df but differ in the Resample Nr.

                                A    B 
   
0     1             0           2.0   3.0
1     1             1           3.0   4.0
2     2             1           3.0   4.0
3     2             1           3.0   4.0
..    ..
..    .. 
39    20            0           2.0    3.0
40    20            0           2.0    3.0

Now I want to apply a certain function on each Reample Nr. Say:

C = sum(df['A'] * df['B']) / sum(df['B'] ** 2)

The outlook would look like this:

                                 A    B           C
0     1             0           2.0   3.0   Calculated Value X1
1     1             1           3.0   4.0   Calculated Value X1
2     2             1           3.0   4.0   Calculated Value X2
3     2             1           3.0   4.0   Calculated Value X2
..    ..
..    .. 
39    20            0           2.0    3.0  Calculated Value  X20
40    20            0           2.0    3.0  Calculated Value  X20

So there are 20 different new values.

I know there is a df.iloc command where I can specify my row selection df.iloc[row, column] but I would like to find a command where I don't have to repeat the code for the 20 samples. My goal is to find a command that identifies the Resample Nr. automatically and then calculates the function for each Resample Nr.

How can I do this?

Thank you!

pandas resampling dataframe python
1 Answers

Use DataFrame.assign to create two new columns x and y that corresponds to df['A'] * df['B'] and df['B']**2, then use DataFrame.groupby on Resample Nr. (or level=1) and transform using sum:

s = df.assign(x=df['A'].mul(df['B']), y=df['B']**2)\
      .groupby(level=1)[['x', 'y']].transform('sum')
df['C'] = s['x'].div(s['y'])

Result:

           A    B         C
0  1  0  2.0  3.0  0.720000
1  1  1  3.0  4.0  0.720000
2  2  1  3.0  4.0  0.750000
3  2  1  3.0  4.0  0.750000
39 20 0  2.0  3.0  0.666667
40 20 0  2.0  3.0  0.666667
user_128331660
Related