Is it possible to reference, or refer back to a previous value in a dataframe column during a vectorised operation?

Viewed 79

I would like to optimize my code using vectorization. Below is a simplified version of the code:

import pandas as pd
import numpy as np

np.random.seed(34)

my_var = 0
def func(num):
    global my_var
    fall = my_var - my_var * 0.3
    if num < fall:
        my_var = num
        return 'MARK'
    elif num > my_var:
        my_var = num
        return 'BLANK'
    else:
        return 'NO CHANGE'
    
    
data = {
    'COL' : np.random.randint(0,10, size=10)
}
 
df = pd.DataFrame(data)

results = df.apply(lambda row: func(row['COL']), 
                   axis = 1)

df['RES'] = results
print(df)

The code above is keeping track of the highest number fed into it from a dataframe, then if the next number passed in is lower by a set percent, the functions returns the 'MARK' string, and the highest number becomes the number passed in. For that, I used a variable to keep track of the current highest number.

I would like to find a way to implement this in a vectorized manner to speed up the execution time of the code as the actual data set I need to process is very large.

I have considered using the .where() or the select() functions to somehow create a vectorized version of the code, but I have no idea how could I go on about achieving this. Is there a way to reference previous values in a dataframe column, or perhaps use variables to store data whilst vectorizing?

Where possible please provide a sample code to demonstrate your suggestion.

Many thanks.

0 Answers
Related