How to apply a function to every element in a dataframe?

user_14828964 0
Viewed 2921

This is probably a very basic question but I can't find the answer in other questions. I have two lists that I have used to create a 2D dataframe, let's say:

X= np.arange(0, 2.01, 0.25)
Y= np.arange(10, 30, 5.0) 

df = pd.DataFrame(index = X, columns = Y)
print(df)

Which gives:

     10.0 15.0 20.0 25.0
0.00  NaN  NaN  NaN  NaN
0.25  NaN  NaN  NaN  NaN
0.50  NaN  NaN  NaN  NaN
0.75  NaN  NaN  NaN  NaN
1.00  NaN  NaN  NaN  NaN
1.25  NaN  NaN  NaN  NaN
1.50  NaN  NaN  NaN  NaN
1.75  NaN  NaN  NaN  NaN
2.00  NaN  NaN  NaN  NaN

I would like to go through all elements in the dataframe and use the values of X and Y as inputs to some function, foo, that I have written. For example, in the 2rd row, 1st column (using zero indexing) position I have (X, Y) = (0.5, 15.0), so in this position I would like to apply foo(0.5, 15.0) and not foo(2, 1).

I think I should be able to use df.apply() or df.applymap() somehow but I can't figure it out!

pandas dataframe numpy python
3 Answers

Since your problem requires access to both the index and column labels of your df you probably want df.apply().

df.apply() has access to a pandas.Series representing each row/column (dependent on axis argument value) and you will have access to the column name and index; whereas df.applymap() utilises each individual value of df at runtime - so you wouldn't necessarily have access to the index and column name as required.

Example

import numpy as np
import pandas as pd 

def foo(name, index):
    return name - index

x = np.arange(0, 2.01, 0.25)
y = np.arange(10, 30, 5.0) 

df = pd.DataFrame(index = x, columns = y)

df.apply(lambda x: foo(x.name, x.index))

Output

       10.0   15.0   20.0   25.0
0.00  10.00  15.00  20.00  25.00
0.25   9.75  14.75  19.75  24.75
0.50   9.50  14.50  19.50  24.50
0.75   9.25  14.25  19.25  24.25
1.00   9.00  14.00  19.00  24.00
1.25   8.75  13.75  18.75  23.75
1.50   8.50  13.50  18.50  23.50
1.75   8.25  13.25  18.25  23.25
2.00   8.00  13.00  18.00  23.00

In the above example the column name and index of each Series constituting df is passed to foo() by way of df.apply(). Within foo() each value is defined by subtracting it's own index value from it's own column name value. Here you can see that the index value for each row is accessed using x.index and the column value is accessed using x.name within the call within df.apply().

Update

Many thanks to @SyntaxError for pointing out that x.index and x.name could be passed to foo() within df.apply() instead of feeding the entire Series (x) into the function and accessing the values manually therein. As mentioned, this seems to fit OP's use case in a much neater manner than my original response - which was largely the same but passed each x series into foo() which then had responsibility for extracting x.name and x.column.

user_120459070

that would be my approach:

from itertools import product

def foo(row, col):
    return row * col

for row, col in product(df.index, df.columns):
    df.loc[row, col] = foo(row, col)

output:

      10.0   15.0 20.0   25.0
0.00     0      0    0      0
0.25   2.5   3.75    5   6.25
0.50     5    7.5   10   12.5
0.75   7.5  11.25   15  18.75
1.00    10     15   20     25
1.25  12.5  18.75   25  31.25
1.50    15   22.5   30   37.5
1.75  17.5  26.25   35  43.75
2.00    20     30   40     50
user_62242460 
    X= np.arange(0, 2.01, 0.25)
    Y= np.arange(10, 30, 5.0) 

    df = pd.DataFrame(index = X, columns = Y)

    # example of function
    def foo(x, y):
        return x*y

    # apply the fonction to each element of the data frame getting the raw and the column name as parameters
    for i in range(df.shape[0]):
    for j in range(df.shape[1]):
        df.iloc[i,j] = foo (float(df.columns[j]),float(df.index[i]))
user_148392530
Related