Rolling and cumulative standard deviation in a Python dataframe

user_2764160 0
Viewed 6753

Is there a vectorized operation to calculate the cumulative and rolling standard deviation (SD) of a Python DataFrame?

For example, I want to add a column 'c' which calculates the cumulative SD based on column 'a', i.e. in index 0, it shows NaN due to 1 data point, and in index 1, it calculates SD based on 2 data points, and so on.

The same question goes to rolling SD too. Is there an efficient way to calculate without iterating through df.itertuples()?

import numpy as np
import pandas as pd

def main():
    np.random.seed(123)
    df = pd.DataFrame(np.random.randn(10, 2), columns=['a', 'b'])
    print(df)

if __name__ == '__main__':
    main()
pandas standard-deviation dataframe python
2 Answers

I think, if by rolling you mean cumulative, then the right term in Pandas is expanding:

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.expanding.html#pandas.DataFrame.expanding

It also accepts a min_periods argument.

df['c'] = df['a'].expanding(2).std()

The case for rolling was handled by Scott Boston, and it is unsurprisingly called rolling in Pandas.

The advantage if expanding over rolling(len(df), ...) is, you don't need to know the len in advance. It is very useful e.g. in groupby dataframes.

user_13387970
Related