Groupby day to show True if identical consecutive values

nipy 0
Viewed 31

Given the following DataFrame. How do I add a new column showing True in next row when two consecutive "y" are seen in a single day in the val column (else False).

  • Each day resets the logic.

  • Essentially looking for two consecutive "y" in a single day then if next row is same day add a True.

Data

df_so = pd.DataFrame(
    {"val": ["y", "y", "y", "n", "y", "y", "y", "y", "n", "n", "y", "y", "n"]},
    index=pd.date_range(start="1/1/2018", periods=13, freq="8h"),
)

df_so

                   val
2018-01-01 00:00:00  y
2018-01-01 08:00:00  y
2018-01-01 16:00:00  y
2018-01-02 00:00:00  n
2018-01-02 08:00:00  y
2018-01-02 16:00:00  y
2018-01-03 00:00:00  y
2018-01-03 08:00:00  y
2018-01-03 16:00:00  n
2018-01-04 00:00:00  n
2018-01-04 08:00:00  y
2018-01-04 16:00:00  y
2018-01-05 00:00:00  n

Desired output would look like this:

                   val  desired_col
2018-01-01 00:00:00 y   False
2018-01-01 08:00:00 y   False
2018-01-01 16:00:00 y   True
2018-01-02 00:00:00 n   False
2018-01-02 08:00:00 y   False
2018-01-02 16:00:00 y   False
2018-01-03 00:00:00 y   False
2018-01-03 08:00:00 y   False
2018-01-03 16:00:00 n   True
2018-01-04 00:00:00 n   False
2018-01-04 08:00:00 y   False
2018-01-04 16:00:00 y   False
2018-01-05 00:00:00 n   False
pandas python
1 Answers

You can use groupby().rolling() here:

target = 2
days = df_so.index.normalize()
df_so['out'] = (df_so['val'].eq('y')
               .groupby(days)
               .rolling(target, min_periods=0).sum()
               .reset_index(level=0,drop=True)
               .groupby(days).shift().eq(target)
          )

Or with groupby().transform:

target = 2
df_so['out'] = (df_so['val'].eq('y')
                    .groupby(df_so.index.normalize())
                    .transform(lambda x: x.rolling(target).sum().shift().eq(target))
               )

Output:

                    val    out
2018-01-01 00:00:00   y  False
2018-01-01 08:00:00   y  False
2018-01-01 16:00:00   y   True
2018-01-02 00:00:00   n  False
2018-01-02 08:00:00   y  False
2018-01-02 16:00:00   y  False
2018-01-03 00:00:00   y  False
2018-01-03 08:00:00   y  False
2018-01-03 16:00:00   n   True
2018-01-04 00:00:00   n  False
2018-01-04 08:00:00   y  False
2018-01-04 16:00:00   y  False
2018-01-05 00:00:00   n  False
Quang Hoang0
Related