Assigning count of unique Year-Month combinations

user_5712858 0
Viewed 372

I want to calculate the number of unique combinations year-months for every respected email

test_df = pd.DataFrame(
    data={'email': ['a', 'a', 'b', 'b', 'c', 'c', 'c'], 
          'purchases': ['2016-08-25 01:09:42',
                        '2016-08-23 13:30:20',
                        '2018-10-23 05:33:15',
                        '2016-09-20 17:41:04',
                        '2017-04-09 17:59:00',
                        '2018-02-25 15:14:53',
                        '2016-02-25 15:14:53']})
test_df['purchases'] = pd.to_datetime(test_df['purchases'], yearfirst=True)

After this, I have this DF with purchases as Timestamps

   email    purchases
0   a   2016-08-25 01:09:42
1   a   2016-08-23 13:30:20
2   b   2018-10-23 05:33:15
3   b   2016-09-20 17:41:04
4   c   2017-04-09 17:59:00
5   c   2018-02-25 15:14:53
6   c   2016-02-25 15:14:53

After this I calculate number of months and assign values to the new column months_of_active:

test_df['months_of_active'] = 
pd.DatetimeIndex(test_df.purchases).to_period("M").nunique()

Which creates the next output:

   email    purchases       months_of_active
0   a   2016-08-25 01:09:42   6
1   a   2016-08-23 13:30:20   6
2   b   2018-10-23 05:33:15   6
3   b   2016-09-20 17:41:04   6
4   c   2017-04-09 17:59:00   6
5   c   2018-02-25 15:14:53   6
6   c   2016-02-25 15:14:53   6

The desired output is:

   email    purchases      months_of_active
0   a   2016-08-25 01:09:42   1
1   a   2016-08-23 13:30:20   1
2   b   2018-10-23 05:33:15   2
3   b   2016-09-20 17:41:04   2
4   c   2017-04-09 17:59:00   3
5   c   2018-02-25 15:14:53   3
6   c   2016-02-25 15:14:53   3

a = 1 because there are two similar months b = 2 because there are two distinct months c = 2 because there are two distinct months (2 the same and 1 another)

Can`t understand, what to add in the function above to perform to_period() on filtered series.

UPDATE: I do need to consider years as well, 2017-1 and 2018-1 will be counted as 2.

pandas dataframe datetime python
2 Answers

You'll need to group on "email" and use transform with nunique to get the unique counts broadcasted to the original DataFrame's rows:

s = pd.Series(pd.DatetimeIndex(df.purchases).to_period('M'), index=df.index)
df['months_of_active'] = s.groupby(df.email).transform('nunique')


df
  email           purchases  months_of_active
0     a 2016-08-25 01:09:42                 1
1     a 2016-08-23 13:30:20                 1
2     b 2018-10-23 05:33:15                 2
3     b 2016-09-20 17:41:04                 2
4     c 2017-04-09 17:59:00                 3
5     c 2018-02-25 15:14:53                 3
6     c 2016-02-25 15:14:53                 3

Alternatively, using dt.strftime to get the Year-Month combination:

df['months_of_active'] = (
   df.purchases.dt.strftime('%Y-%m').groupby(df.email).transform('nunique'))

df
  email           purchases  months_of_active
0     a 2016-08-25 01:09:42                 1
1     a 2016-08-23 13:30:20                 1
2     b 2018-10-23 05:33:15                 2
3     b 2016-09-20 17:41:04                 2
4     c 2017-04-09 17:59:00                 3
5     c 2018-02-25 15:14:53                 3
6     c 2016-02-25 15:14:53                 3
user_49090870

To avoid conversion to year-month strings or object dtype series you can normalize day and time components of your datetime series, then use pd.Series.nunique:

# convert purchases series to datetime
df['purchases'] = pd.to_datetime(df['purchases'])

# normalize day to 1 and zero time component
df['year_month'] = (df['purchases'] + pd.offsets.MonthBegin(1)).dt.normalize()

# calculate counts
email_counts = df.groupby('email')['year_month'].nunique()

# assign counts to series and drop helper series
df = df.assign(count=df['email'].map(email_counts)).drop('year_month', 1)

print(df)

  email           purchases  count
0     a 2016-08-25 01:09:42      1
1     a 2016-08-23 13:30:20      1
2     b 2018-10-23 05:33:15      2
3     b 2016-09-20 17:41:04      2
4     c 2017-04-09 17:59:00      3
5     c 2018-02-25 15:14:53      3
6     c 2016-02-25 15:14:53      3
user_92095460
Related