How to create a increment var from a first value of a dataframe group?

user_11755414 0
Viewed 89

I have a datframe as :

data=[[0,1,5],
      [0,1,6],
      [0,0,8],
      [0,0,10],
      [0,1,12],
      [0,0,14],
      [0,1,16],
      [0,1,18],    
      [1,0,2],
      [1,1,0],
      [1,0,1],
      [1,0,2]]

df = pd.DataFrame(data,columns=['KEY','COND','VAL'])

For RES1, I want to create a counter variable RES where COND ==1. The value of RES for the first KEY of the group remains same as the VAL (Can I use cumcount() in some way).

For RES2, then I just want to fill the missing values as the previous value. (df.fillna(method='ffill')), I am thinking..

    KEY  COND    VAL RES1  RES2
0     0     1    5      5     5
1     0     1    6      6     6
2     0     0    8            6
3     0     0   10            6
4     0     1   12      7     7
5     0     0   14            7
6     0     1   16      8     8
7     0     1   18      9     9
8     1     0    2      2     2
9     1     1    0      3     3
10    1     0    1            3
11    1     0    2            3

Aim is to look fir a vectorized solution that's most optimal over million rows.

pandas numpy python
2 Answers

IIUC

con=(df.COND==1)|(df.index.isin(df.drop_duplicates('KEY').index))
df['res1']=df.groupby('KEY').VAL.transform('first')+
           df.groupby('KEY').COND.cumsum()[con]-
           df.groupby('KEY').COND.transform('first')
df['res2']=df.res1.ffill()
df
   Out[148]: 
    KEY  COND  VAL  res1  res2
0     0     1    5   5.0   5.0
1     0     1    6   6.0   6.0
2     0     0    8   NaN   6.0
3     0     0   10   NaN   6.0
4     0     1   12   7.0   7.0
5     0     0   14   NaN   7.0
6     0     1   16   8.0   8.0
7     0     1   18   9.0   9.0
8     1     0    2   2.0   2.0
9     1     1    0   3.0   3.0
10    1     0    1   NaN   3.0
11    1     0    2   NaN   3.0
user_79645270

You want:

s = (df[df.KEY.duplicated()]          # Ignore first row in each KEY group
        .groupby('KEY').COND.cumsum() # Counter within KEY
        .add(df.groupby('KEY').VAL.transform('first')) # Add first value per KEY
        .where(df.COND.eq(1))                          # Set only where COND == 1
        .add(df.loc[~df.KEY.duplicated(), 'VAL'], fill_value=0) # Set 1st row by KEY
     )
df['RES1'] = s
df['RES2'] = df['RES1'].ffill()

    KEY  COND  VAL  RES1  RES2
0     0     1    5   5.0   5.0
1     0     1    6   6.0   6.0
2     0     0    8   NaN   6.0
3     0     0   10   NaN   6.0
4     0     1   12   7.0   7.0
5     0     0   14   NaN   7.0
6     0     1   16   8.0   8.0
7     0     1   18   9.0   9.0
8     1     0    2   2.0   2.0
9     1     1    0   3.0   3.0
10    1     0    1   NaN   3.0
11    1     0    2   NaN   3.0
user_43333590
Related