creating new variable that takes into account prior information from the earlier records

A solution using dplyr. We can remove duplicated rows based on moviewatched and name, count unique moviewatched, and then use cumsum to calcualte the running total. df2 is the final output.

library(dplyr)

df2 <- df %>%
  distinct(moviewatched, name, .keep_all = TRUE) %>%
  group_by(name, time) %>%
  summarise(movietypewatched = n_distinct(moviewatched)) %>%
  mutate(movietypewatched = cumsum(movietypewatched)) %>%
  ungroup()
df2
# # A tibble: 4 x 3
#   name  time   movietypewatched
#   <fct> <fct>             <int>
# 1 john  1-2018                2
# 2 john  2-2018                3
# 3 kate  1-2018                1
# 4 kate  2-2018                2

And here is a data.table solution following the same logic.

library(data.table)

setDT(df)
df2 <- df[!duplicated(df[, .(moviewatched, name)])][
  , .(movietypewatched = uniqueN(moviewatched)), by = .(name, time)][
    , movietypewatched := cumsum(movietypewatched), by = name]
df2[]
#    name   time movietypewatched
# 1: john 1-2018                2
# 2: john 2-2018                3
# 3: kate 1-2018                1
# 4: kate 2-2018                2

First convert the time data to a class to establish order, e.g. with lubridate::myd with truncated = 1. From here, set the arrangement of rows to ensure they're in order, then, grouped by name, use purrr::accumulate to generate a list of unique values seen so far in moviewatched, called upon which lengths will return the number of movies seen to that point. Aggregate by month with max to get the total cumulative types for each month.

library(tidyverse)

df <- data_frame(
    moviewatched =  c('Comedy', 'Horror', 'Comedy', 'Horror', 'Drama', 'Comedy', 'Drama'),
    name =  c('john', 'john', 'john', 'john', 'john','kate','kate'),
    time =  lubridate::myd(c('1-2018', '1-2018', '1-2018', '2-2018', '2-2018','1-2018' ,'2-2018'), truncated = 1)
)

df %>% 
    group_by(name) %>% 
    arrange(name, time) %>%
    mutate(n_types = lengths(accumulate(moviewatched, ~unique(c(...))))) %>% 
    group_by(name, time) %>% 
    summarise(n_types = max(n_types))
#> # A tibble: 4 x 3
#> # Groups:   name [2]
#>   name  time       n_types
#>   <chr> <date>       <dbl>
#> 1 john  2018-01-01       2
#> 2 john  2018-02-01       3
#> 3 kate  2018-01-01       1
#> 4 kate  2018-02-01       2

Make a table of first dates watched; count by month; and take the cumulative sum:

library(data.table)
setDT(df)

# fix bad date
df[, d := as.IDate(paste(time, "01", sep="-"), "%m-%Y-%d")]

# identify month first watched
fw = df[, .(d = min(d)), by=.(name, moviewatched)]

# count new movies per month
nm = fw[, .N, keyby=.(name, d)]

# take cumulative count
nm[, cN := cumsum(N), by=name]

   name          d N cN
1: john 2018-01-01 2  2
2: john 2018-02-01 1  3
3: kate 2018-01-01 1  1
4: kate 2018-02-01 1  2

You need to convert the date; otherwise the min() will be incorrect and/or broken.

There are two aggregation steps here, but the code should be fast thanks to optimization in data.table (see ?GForce).

Using data.table:

library(data.table)
df <- unique(df) 
setDT(df)[, movietypewatched := 1:.N, by = c("moviewatched", "name")] 
df <- df[!(movietypewatched == 2), ]
df[, movietypewatched := .N, by = c("name", "time")][, moviewatched := NULL]
df <- unique(df)
df[, movietypewatched := cumsum(movietypewatched), by = name]

   name   time movietypewatched
1: john 1-2018                2
2: john 2-2018                3
3: kate 1-2018                1
4: kate 2-2018                2

Here, you can make intermediate steps if you'd like to get the unique values in genre_all and the count in genre_count.

Note that:

• You need to arrange the dataframe by name, date to accumulate values.
• You can use lag() to get the previous value. Since the first entry for each name has no previous value, it will give NA.
• You will need to remove the NAs when you count the unique genres using n_distinct().

>

library(dplyr)
library(purrr)
library(tidyr)

moviewatched <- c('Comedy', 'Horror', 'Comedy', 'Horror', 'Drama', 'Comedy', 'Drama')
name <- c('john', 'john', 'john', 'john','kate','kate', 'john')
time <- c( '1-2018', '1-2018', '2-2018', '2-2018','1-2018' ,'2-2018','1-2018')

df <- data.frame(moviewatched, name, time)


df_final <- df %>% 
  arrange(name, time) %>% 
  group_by(name, time) %>%
  nest(.key= 'genre') %>% 
  group_by(name) %>% 
  mutate(genre_all = map2(genre, lag(genre), rbind) %>% map(unique)) %>% 
  ungroup() %>% 
  mutate(genre_count = map_int(genre_all, ~ lift(n_distinct)(.x, na.rm =TRUE)))

Result:

> df_final
# A tibble: 4 x 5
  name  time   genre            genre_all        genre_count
  <fct> <fct>  <list>           <list>                 <int>
1 john  1-2018 <tibble [3 x 1]> <tibble [3 x 1]>           2
2 john  2-2018 <tibble [2 x 1]> <tibble [3 x 1]>           3
3 kate  1-2018 <tibble [1 x 1]> <tibble [2 x 1]>           1
4 kate  2-2018 <tibble [1 x 1]> <tibble [2 x 1]>           2