R datatable grouping based on condition and getting count based on the conditions

user_13566273 0
Viewed 159

I have a data table like this :

timestamp           type    status
05-01-2020 12:07:08    A      1
05-01-2020 12:36:05    A      1 
05-01-2020 13:34:25    A      1 
05-01-2020 23:45:02    A      1
05-01-2020 23:55:02    B      1
05-01-2020 13:44:33    B      2
06-01-2020 01:07:08    A      1 
06-01-2020 10:23:05    A      1
06-01-2020 12:11:08    A      2
06-01-2020 22:06:12    B      2
07-01-2020 00:01:05    A      2
07-01-2020 02:17:09    A      1
07-01-2020 12:36:05    B      1
07-01-2020 12:07:08    B      1
07-01-2020 12:36:05    A      1
07-01-2020 12:36:05    A      1
08-01-2020 12:36:05    B      2
08-01-2020 12:36:05    B      1
08-01-2020 12:36:05    B      1
09-01-2020 12:36:05    B      1 
09-01-2020 12:07:08    B      2
09-01-2020 12:36:05    B      1
11-01-2020 12:07:08    A      1
11-01-2020 12:36:05    A      1

I am trying to group it on date and type using rleid().

dt <- dt[, group_id := rleid(as.IDate(timestamp),type,status = 1)][]

Now I want to get two counts.

One is to count the number of instances inside each group which meets the condition per day.

date         type  count
05-01-2020    A      4
05-01-2020    B      1
06-01-2020    A      2
07-01-2020    A      3
07-01-2020    B      2
08-01-2020    B      2
09-01-2020    B      2
11-01-2020    A      2

Second one is to find the number of groups per day which meet the condition.

date         type  count
05-01-2020    A      1
05-01-2020    B      1
06-01-2020    A      1
07-01-2020    A      2
07-01-2020    B      1
08-01-2020    B      1
09-01-2020    B      2
11-01-2020    A      1
data.table r
2 Answers

1) To count the number of instances inside each group which meets the condition per day. 

library(data.table)
setDT(df)
df[, .(count = sum(status == 1)), .(timestamp, type)]

#    timestamp type count
#1: 05-01-2020    A     4
#2: 05-01-2020    B     1
#3: 06-01-2020    A     2
#4: 06-01-2020    B     0
#5: 07-01-2020    A     3
#6: 07-01-2020    B     2
#7: 08-01-2020    B     2
#8: 09-01-2020    B     2
#9: 11-01-2020    A     2

You can remove the 0 counts if they are not needed.

2) To find the number of groups per day which meet the condition.

Create a new column (count_N) using rleid of type and status and for status = 1 count unique values for each timestamp and type.

df[, count_N := rleid(type, status), timestamp]
df[status == 1, .(count = uniqueN(count_N)), .(timestamp, type)]


#    timestamp type count
#1: 05-01-2020    A     1
#2: 05-01-2020    B     1
#3: 06-01-2020    A     1
#4: 07-01-2020    A     2
#5: 07-01-2020    B     1
#6: 08-01-2020    B     1
#7: 09-01-2020    B     2
#8: 11-01-2020    A     1
user_39629140

We can first convert the 'timestamp' to Datetime class with as.POSIXct and then convert it to Date class

library(data.table)
setDT(dt)[, timestamp := as.POSIXct(timestamp, 
     format = '%m-%d-%Y %H:%M:%S')][, date := as.IDate(timestamp)]
dt[status == 1, .N, .(date, type)]
#.        date type N
#1: 2020-05-01    A 4
#2: 2020-05-01    B 1
#3: 2020-06-01    A 2
#4: 2020-07-01    A 3
#5: 2020-07-01    B 2
#6: 2020-08-01    B 2
#7: 2020-09-01    B 2
#8: 2020-11-01    A 2

For the second case

dt[, grp := rleid(type, status, date)]
dt[status == 1, .(count = uniqueN(grp)), .(date, type)]
#         date type count
#1: 2020-05-01    A     1
#2: 2020-05-01    B     1
#3: 2020-06-01    A     1
#4: 2020-07-01    A     2
#5: 2020-07-01    B     1
#6: 2020-08-01    B     1
#7: 2020-09-01    B     2
#8: 2020-11-01    A     1

data

dt <- structure(list(timestamp = structure(c(1588349228, 1588350965, 
1588354465, 1588391102, 1588391702, 1588355073, 1590988028, 1591021385, 
1591027868, 1591063572, 1593576065, 1593584229, 1593621365, 1593619628, 
1593621365, 1593621365, 1596299765, 1596299765, 1596299765, 1598978165, 
1598976428, 1598978165, 1604250428, 1604252165), class = c("POSIXct", 
"POSIXt"), tzone = ""), type = c("A", "A", "A", "A", "B", "B", 
"A", "A", "A", "B", "A", "A", "B", "B", "A", "A", "B", "B", "B", 
"B", "B", "B", "A", "A"), status = c(1L, 1L, 1L, 1L, 1L, 2L, 
1L, 1L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L, 2L, 1L, 1L, 1L, 2L, 1L, 
1L, 1L)), class = "data.frame", row.names = c(NA, -24L), 
index = structure(integer(0), "`__status`" = c(1L, 
2L, 3L, 4L, 5L, 7L, 8L, 12L, 13L, 14L, 15L, 16L, 18L, 19L, 20L, 
22L, 23L, 24L, 6L, 9L, 10L, 11L, 17L, 21L)))
user_37322710
Related