The best way to compare a value in a dataframe row with the previous row

user_2649644 0
Viewed 62

I have a rather large df in which I need to take the difference of one column to the value of a different column in the previous row. I've tried to use iterrows and itertuples (and surprisingly iterrows was slightly quicker) but it's not nearly as quick as it should be. I was wondering if there was a much more efficient way to complete this task.

Here's what I have so far, but I can't seem to find a better way as of yet.

def make_decision(df, max_gap):


    make_decision = []
    new_decision = []

    df.sort_values(by=["start_time"])
    tod = df.iloc[0]["stop_time"]

    for row in df.itertuples(index=False):

        toa = row[df.columns.get_loc("start_time")]

        if toa - tod > max_gap:
            make_decision.append(row[df.columns.get_loc("id")])
            new_decision.append(make_decision)
            make_decision.clear()

        else:
            make_decision.append(row[df.columns.get_loc("id")])

        curr_tod = row[df.columns.get_loc("stop_time")]

        if tod < curr_tod:
            tod = curr_tod
       
    if make_decision:
        new_decision.append(make_decision)

    return new_decision
pandas dataframe python
1 Answers

Depending on direction, you can use:

df['start_time'] - df.shift()['stop_time']

or

df['start_time'] - df.shift(-1)['stop_time']

or interchange start_time and stop_time.

user_64014030
Related