Extracting the steps or increments within a list in a Series

user_8480857 0
Viewed 45

I'm trying to set a new column in a DataSeries based on the difference between each value contained in lists in different a column. The increments in those lists are 1, 2 or 5. For example, in a list like [23,24,25,26,27,28], I'd like to return 1, because that's the step/difference between each value.

import pandas as pd

test = pd.Series([23,24,25,26,27,28])
changes = []
for x1, x2 in zip(test[:-1], test[1:]):
    changes.append(x2 - x1)

print(changes)

-> output: [1, 1, 1, 1, 1,]

then I get the number of unique values in this list:

list_set = set(changes)
print(list_set)

and I get my answer:

output: {1}

The problem I have is to do that within a DataFrame, with the list within a value:

test              changes                   
test03                            [3, 8]    
test45                            [0, 4]    
test07    [7, 8, 9, 0, 1, 2, 3, 4, 5, 6]    
test04                   [2, 4, 6, 8, 0]    
test09    [0, 1, 9, 2, 3, 4, 5, 6, 7, 8]

I'd like to get something like:

test              changes                   steps
test03                            [3, 8]    5
test45                            [0, 4]    5
test07    [7, 8, 9, 0, 1, 2, 3, 4, 5, 6]    1
test04                   [2, 4, 6, 8, 0]    2
test09    [0, 1, 9, 2, 3, 4, 5, 6, 7, 8]    1

I'd like to sort each lists, then apply the same, but I can't figure out how to run it within each value lists

pandas list python
2 Answers

Try using apply and diff:

df['steps'] = df['changes'].apply(lambda x: pd.Series(sorted(x)).diff().max())

And now:

print(df)

Is:

    test                         changes  steps
0  test03                          [3, 8]    5.0
1  test45                          [0, 4]    4.0
2  test07  [7, 8, 9, 0, 1, 2, 3, 4, 5, 6]    1.0
3  test04                 [2, 4, 6, 8, 0]    2.0
4  test09  [0, 1, 9, 2, 3, 4, 5, 6, 7, 8]    1.0
user_87083640

You can achieve it through applying numpy functions np.sort, np.ediff1d and np.max for fast execution, as follows:

import numpy as np

df['steps'] = df['changes'].apply(lambda x: np.max(np.ediff1d(np.sort(x))))

Result:

print(df)

     test                         changes  steps
0  test03                          [3, 8]      5
1  test45                          [0, 4]      4
2  test07  [7, 8, 9, 0, 1, 2, 3, 4, 5, 6]      1
3  test04                 [2, 4, 6, 8, 0]      2
4  test09  [0, 1, 9, 2, 3, 4, 5, 6, 7, 8]      1
user_150706970
Related