I have a pandas.Series of positive numbers. I need to find the indexes of "outliers", whose values depart by 3 or more from the previous "norm".
How to vectorize this function:
def baseline(s):
values = []
indexes = []
last_valid = s.iloc[0]
for idx, val in s.iteritems():
if abs(val - last_valid) >= 3:
values.append(val)
indexes.append(idx)
else:
last_valid = val
return pd.Series(values, index=indexes)
For example, if the input is:
import pandas as pd
s = pd.Series([7,8,9,10,14,10,10,14,100,14,10])
print baseline(s)
the desired output is:
4 14
7 14
8 100
9 14
Note that the 10 values after the 14s are not returned because they are "back to normal" values.
Edits:
- Added
abs()to the code. The numbers are positive. - The purpose here is to speed up the code.
- An answer that doesn't exactly imitate the code may be acceptable.
- Changed the example to include another edge case, where the values slowly change by 3.