pandas .diff by multiindex values of rows

user_1233751 0
Viewed 641
In [66]: t1
Out[69]: 
job_date  branch_id
2018-05   1            0.618980
          2            0.600590
          3            0.603486
          4            0.043931
          5            0.588168
          6            0.381518
          7            0.357035
2018-06   1            0.690575
          2            0.700900
          3            0.571556
          4            0.351935
          5            0.626428
          6            0.461813
          7            0.329663
Name: utilization, dtype: float64

In [86]: t1.index
Out[86]: 
MultiIndex(levels=[[2018-05, 2018-06], [1, 2, 3, 4, 5, 6, 7]],
           labels=[[0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1], [0, 1, 2, 3, 4, 5, 6, 0, 1, 2, 3, 4, 5, 6]],
           names=['job_date', 'branch_id'])

How to make a diff on rows by index values?

So diff between (2018-05, 1) and (2018-06, 1) should be 0.690575-0.618980=0.071595

If I do t1.diff(), I get row by row comparison, which is not what I want

In [87]: t1.diff()
Out[87]: 
job_date  branch_id
2018-05   1                 NaN
          2           -0.018390
          3            0.002895
          4           -0.559554
          5            0.544237
          6           -0.206651
          7           -0.024483
2018-06   1            0.333540
          2            0.010325
          3           -0.129345
          4           -0.219621
          5            0.274494
          6           -0.164615
          7           -0.132150

Right now I'm doing this

In [49]: t1.unstack(level=0)['utilization'].diff(axis=1)
Out[49]: 
job_date   2018-05   2018-06
branch_id                   
1              NaN  0.071595
2              NaN  0.100310
3              NaN -0.031930
4              NaN  0.308003
5              NaN  0.038260
6              NaN  0.080295
7              NaN -0.027372

is there a way without unstacking??

pandas-groupby pandas python
2 Answers

One possible solution is shift MultiIndex by one month and subtract, it working if difference between each Period is same - here one month:

a = df.index.get_level_values(0).to_period('M')
b = df.index.get_level_values(1)
mux1 = pd.MultiIndex.from_arrays([a,b], names=df.index.names)
mux2 = pd.MultiIndex.from_arrays([a + 1, b], names=df.index.names)

df = df.set_index(mux1)
df1 = df.set_index(mux2)

df['utilization'] = df.sub(df1)
print (df)
                    utilization
job_date branch_id             
2018-05  1                  NaN
         2                  NaN
         3                  NaN
         4                  NaN
         5                  NaN
         6                  NaN
         7                  NaN
2018-06  1             0.071595
         2             0.100310
         3            -0.031930
         4             0.308004
         5             0.038260
         6             0.080295
         7            -0.027372
user_29010020

You can use groupby without unstacking like so

import pandas as pd

ix = pd.MultiIndex(
    levels=[['2018-05', '2018-06'], [1, 2, 3, 4, 5, 6, 7]],
    labels=[[0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1],
            [0, 1, 2, 3, 4, 5, 6, 0, 1, 2, 3, 4, 5, 6]],
    names=['job_date', 'branch_id'])
​

series = pd.Series(
    [0.618980, 0.600590, 0.603486, 0.043931, 0.588168, 0.381518,
     0.357035, 0.690575, 0.700900, 0.571556, 0.351935, 0.626428,
     0.461813, 0.329663], 
    index=ix)

series.groupby(by='branch_id').diff()

Output:

job_date  branch_id
2018-05   1                nan
          2                nan
          3                nan
          4                nan
          5                nan
          6                nan
          7                nan
2018-06   1            0.07160
          2            0.10031
          3           -0.03193
          4            0.30800
          5            0.03826
          6            0.08029
          7           -0.02737
dtype: float64
user_19489820
Related