identify blocks of consecutive True values with tolerance

user_1506850 0
Viewed 335

I have a boolean pandas DataFrame:

w=pd.DataFrame(data=[True,False,True,True,True,False,False,True,False,True,True,False,True])

I am trying to identify the blocks of True values, which are long at least N:

I can do that (as suggested elsewere on SO) by

N=3.0
b = w.ne(w.shift()).cumsum() *w
m = b[0].map(b[0].mask(b[0] == 0).value_counts()) >= N

which works fine and returns 

m
0     False
1     False
2      True
3      True
4      True
5     False
6     False
7     False
8     False
9     False
10    False
11    False
12    False

Now, I need to do the same buyt allow for some tolerance in determining the blocks. so I would like to identify all the blocks long at least N, but allowing for M values (arbitrarly placed within the block) to be False.

For the example w, N=3, and M=1 it should be,

w
0    True
1   False
2    True
3    True
4    True
5   False
6   False
7    True
8   False
9    True
10   True
11  False
12   True

differently from previous results at:

desidered=
0    **True**
1    **True**
2    True
3    True
4    True
5   False
6   False
7    True
8  ** True **
9    True
10   True
11   **True**
12   True
pandas boolean python
1 Answers

I believe you can re-use solution with inverting m by ~ and last chain both conditions by or :

N = 3.0
M = 1
b = w.ne(w.shift()).cumsum() *w
m = b[0].map(b[0].mask(b[0] == 0).value_counts()) <= N

w1 = ~m
b1 = w1.ne(w1.shift()).cumsum() * w1
m1 = b1.map(b1.mask(b1 == 0).value_counts()) == M 

m = m | m1

print (m)
0      True
1      True
2      True
3      True
4      True
5     False
6     False
7      True
8      True
9      True
10     True
11     True
12     True
Name: 0, dtype: bool
user_29010020
Related