I have a dataframe
x y
a 1
b 1
c 1
d 0
e 0
f 0
g 1
h 1
i 0
j 0
I want to remove the rows with 0 except every first new occurence of 0 after 1, so the resultant dataframe should be
x y
a 1
b 1
c 1
d 0
g 1
h 1
i 0
Is it possible to do it without creating groups or row by row iteration to make it faster since I have a big dataframe.
Check consecutive similarity using shift()
df[df.y.ne(0)|(df.y.eq(0)&df.y.shift(1).ne(0))]
x y
0 a 1
1 b 1
2 c 1
3 d 0
6 g 1
7 h 1
8 i 0
Let us try diff with cumsum create the continue value group , then try duplicated
out = df[~df.y.diff().ne(0).cumsum().duplicated() | df.y].copy()
Out[352]:
x y
0 a 1
1 b 1
2 c 1
3 d 0
6 g 1
7 h 1
8 i 0