R: create a new df with conditional statement (related to row n and row n-1)

Viewed 92

I have a data frame with 1 column for participants and one column for my eeg triggers. Example:

ID trigger
P1 SB
P1 SB
P1 resp
P1 DH
P1 Sc
P1 resp
P2 SB
P2 resp
P2 Sc
P2 SB

"resp" correspond to each time the participant has answered (pressed a button). If he answered after Sc, it is a hit, if he don't, it is a miss; if he answered after something else, it is a false alarm (fa); if he does not answer after something which is not Sc, it is a correct rejection (cr).

I would like to create a new data frame and to have, for each participant the total number of Sc, the number of hit, of miss, of fa and of cr, like the following:

ID Nb_Sc hit miss fa cr
P1 100 99 1 1 99
P2 50 45 5 3 47

But i don't know at all how I could do that. Does anyone has an idea and can help?

Thanks for reading.

4 Answers

I propose this simple solution.

  • All the triggers may be divided into two parts actual triggers say Atrigger (i.e. != 'resp') or ACTION and responses where trigger == 'resp' REACTION
  • participants can respond on Atrigger ; so Atrigger + resp equals total row count per participant i.e. n()
  • Now each Atrigger can be divided into two parts - negatives and positives ; positives where participant responded and negative where s/he didn't
  • each positive & negative can be T/F according to the condition if action is true or false
  • Nb_Sc is simply sum of Sc per group
  • hit is simply sum of resp preceded by Sc TRUE POSITIVES
  • miss is Nb_Sc - hit (if I am not wrong) FALSE NEGATIVE
  • fa is sum of resp not preceded by Sc (Again, if I am correct, sum of fa and hit should equal to number of resp per group) FALSE POSITIVES
  • cr is actual triggers - sum(hit + miss + fa) i.e. TRUE NEGATIVES
df = structure(list(ID = c("P1", "P1", "P1", "P1", "P1", "P1", "P1", 
                           "P1", "P2", "P2", "P2", "P2"), trigger = c("SB", "SB", "resp", 
                                                                      "DH", "Sc", "resp", "Sc", "resp", "SB", "resp", "Sc", "SB")), class = "data.frame", row.names = c(NA, 
                                                                                                                                                                        -12L))
library(dplyr)

df %>%
  group_by(ID) %>%
  summarise(Nb_Sc = sum(trigger == 'Sc'),
            hit = sum(trigger == 'resp' & lag(trigger) == 'Sc'),
            miss = Nb_Sc - hit,
            fa = sum(trigger == 'resp' & lag(trigger) != 'Sc'),
            cr = n() - (sum(trigger == 'resp') + hit + miss + fa))

# A tibble: 2 x 6
  ID    Nb_Sc   hit  miss    fa    cr
  <chr> <int> <int> <int> <int> <int>
1 P1        2     2     0     1     2
2 P2        1     0     1     1     1

On the data by Sinh it gives this answer

# A tibble: 2 x 6
  ID    Nb_Sc   hit  miss    fa    cr
  <chr> <int> <int> <int> <int> <int>
1 P1       21     4    17    23    25
2 P2       25     6    19    23    27
library(dplyr)
set.seed(100)
df <- data.frame(
  ID = sample(c("P1", "P2"), 200, replace = TRUE),
  trigger = sample(c("SB", "Sc", "resp", "DH"), 200, replace = TRUE)
)
# Instead of look into previous row I look ahead based on value of current rows
df <- df %>%
  # added group_by here for correct calculated lead / lag
  group_by(ID) %>%
  mutate(category = case_when(
    # hit for all trigger ending with c follow by a resp
    grepl(".+c$", trigger) & lead(trigger, 1) == "resp" ~ "hit",
    # miss for all trigger ending with c not follow by a resp
    grepl(".+c$", trigger) & lead(trigger, 1) != "resp" ~ "miss",
    # fa for all trigger that not ending with c and follow b a resp
    !grepl("(.+c$)|(^resp$)", trigger) & lead(trigger, 1) == "resp" ~ "fa",
    # cr for all trigger that not ending with c and not follow by a resp
    !grepl("(.+c$)|(^resp$)", trigger) & lead(trigger, 1) != "resp" ~ "cr",
    TRUE ~ "No categorized yet"
  ))

# Here is what the data look like
head(df)
#> # A tibble: 6 x 3
#> # Groups:   ID [2]
#>   ID    trigger category          
#>   <chr> <chr>   <chr>             
#> 1 P2    Sc      miss              
#> 2 P1    Sc      miss              
#> 3 P2    SB      fa                
#> 4 P2    resp    No categorized yet
#> 5 P1    Sc      miss              
#> 6 P1    SB      cr

# all no categorized yet is for resp trigger which is accurate.
table(df$trigger, df$category)
#>       
#>        cr fa hit miss No categorized yet
#>   DH   39 14   0    0                  0
#>   resp  0  0   0    0                 56
#>   SB   27 18   0    0                  0
#>   Sc    0  0  10   36                  0
# now summarized the data by ID
df %>%
  group_by(ID) %>%
  summarize(Nb_Sc = sum(trigger == "Sc"),
    hit = sum(category == "hit"),
    miss = sum(category == "miss"),
    fa = sum(category == "fa"),
    cr = sum(category == "cr"))
#> # A tibble: 2 x 6
#>   ID    Nb_Sc   hit  miss    fa    cr
#>   <chr> <int> <int> <int> <int> <int>
#> 1 P1       21     4    17    15    33
#> 2 P2       25     6    19    17    33

Created on 2021-06-15 by the reprex package (v2.0.0)

Updated the solution to match by regex

You could do it with a simple for-loop in a function:

scan.reactions = function(df){
  out.id = c()
  out.nsc = c()
  out.hit = c()
  out.miss = c()
  out.fa = c()
  out.cr = c()
  for(resp in unique(df[,1])){
    out.id = c(out.id,resp)
    seq = c('',df[df[,1]==resp,2]) ## relevant sequence of events, leading with none
    out.nsc = c(out.nsc,sum(seq=="Sc"))
    lseq = c(df[df[,1]==resp,2],'')
    t = as.matrix(table(seq=="Sc",lseq=="resp")) ## Cross table of Sc and subsequent resp
    out.hit = c(out.hit,t[2,2]) ## True/True
    out.miss = c(out.miss,t[2,1])
    out.fa = c(out.fa,t[1,2])
    out.cr = c(out.cr,t[1,1])
  }
  return(data.frame("ID"=out.id,
                    "Nb_Sc"=out.nsc,
                    "hit"=out.hit,
                    "miss"=out.miss,
                    "fa"=out.fa,
                    "cr"=out.cr))
}

I changed your data slightly, so that there is one more correct reaction for P1 and tested it:

> df = structure(list(ID = c("P1", "P1", "P1", "P1", "P1", "P1", "P1", 
"P1", "P2", "P2", "P2", "P2"), trigger = c("SB", "SB", "resp", 
"DH", "Sc", "resp", "Sc", "resp", "SB", "resp", "Sc", "SB")), class = "data.frame", row.names = c(NA, 
-12L))

> df
   ID trigger
1  P1      SB
2  P1      SB
3  P1    resp
4  P1      DH
5  P1      Sc
6  P1    resp
7  P1      Sc
8  P1    resp
9  P2      SB
10 P2    resp
11 P2      Sc
12 P2      SB

> print(scan.reactions(df))
  ID Nb_Sc hit miss fa cr
1 P1     2   2    0  1  6
2 P2     1   0    1  1  3

It uses a cross table of Sc and subsequent resp and seems to produce the correct results.

Here is an alternative, data-driven approach which uses a two dimensional lookup table to describe the category of two consecutive triggers. Subsequently, observed data are joined with the lookup table to categorize and count the sequence of events.

This approach

  • avoids hardcoded case_when() statements or summation of hardcoded conditions
  • covers all possible combinations of two consecutive triggers thereby exhibiting gaps in the definition of categories
  • is easily adapted to different definitions of categories

From the question and OP's comment I understand that all values in trigger fall into one of three groups:

  1. resp = participant has answered
  2. Sc = triggers ending with the letter c like Sc, or SBc, L1c, H1c, etc
  3. other = all other triggers which are neither resp nor ending with the letter c

With this simplification, we can define the categories of consecutive triggers as shown in the 2-D lookup table below:

other Sc resp
other cr cr fa
Sc miss miss hit
resp NA NA NA

Rows refer to the preceding trigger and columns to the subsequent trigger. So, other followed by resp belongs to category fa (false alarm).

NA indicates that this sequence is not categorized yet and should not be counted. The OP has not explicitely specified how to categorize trigger sequences after resp so I have deliberately chosen to ignore them to be in line with Sinh Nguyen's answer. But see below for how to modify the categorisation easily.

library(data.table)
# define categories
lut2D <- fread(
  "t1    other Sc   resp
   other cr    cr   fa
   Sc    miss  miss hit
   resp  NA    NA   NA"
)
# reshape to long format
lut1D <- 
  melt(lut2D, id.var = "t1", variable.name = "t2", value.name = "category", na.rm = TRUE)[
    # order categories as expected by the OP
    , category := factor(category, levels = c("hit", "miss", "fa", "cr", "dr"))][]
# prepare trigger values
tmp <- as.data.table(df)[
  # re-group trigger values: all values ending with "c" 
  trigger %like% "c$", trigger := "Sc"][
    # re-group trigger values: all other values 
    !trigger %in% c("Sc", "resp"), trigger := "other"][
      # create overlapping subsequences of length 2 in wide format 
      # and summaries per ID
      , .(t1 = head(trigger, -1L), t2 = tail(trigger, -1L),
          Nb_trig = .N, Nb_Sc = sum(trigger == "Sc")), keyby = ID]
# right join with lookup table to get matching categories, drop undefined categories
categorised <- lut1D[tmp, on = .(t1, t2), nomatch = NULL]
# count categories
dcast(categorised, ID + Nb_trig + Nb_Sc ~ category, fun = length)

With Sinh Nguyen's data we get

   ID Nb_trig Nb_Sc hit miss fa cr
1: P1      96    21   4   17 15 33
2: P2     104    25   6   19 17 33

Please, note that I have added a count of trigger values Nb_trig per ID. So for P1 96 trigger values were recorded which yield 95 subsequences of length 2.

Additional explanations

  • I have used data.table as I am more acquainted with it. dplyr & tidyr may work as well.
  • The code uses data.table's command chaining.
  • The categorisation is applied on subsequences of length 2. In order to join the observed data and the lookup table both datasets have to be reshaped so that they include a column t1 for the preceding trigger value and a column t2 for the subsequent trigger value.
  • Thus, the reshaped lookup table lut1d looks as follows
      t1    t2 category
1: other other       cr
2:    Sc other     miss
3: other    Sc       cr
4:    Sc    Sc     miss
5: other  resp       fa
6:    Sc  resp      hit
  • and the reshaped observed dataset tmp have become
     ID    t1    t2 Nb_trig Nb_Sc
  1: P1    Sc    Sc      96    21
  2: P1    Sc other      96    21
  3: P1 other    Sc      96    21
  4: P1    Sc other      96    21
  5: P1 other other      96    21
 ---                             
194: P2 other other     104    25
195: P2 other other     104    25
196: P2 other  resp     104    25
197: P2  resp other     104    25
198: P2 other  resp     104    25
  • After the join, the categorised observed dataset categorised is
        t1    t2 category ID Nb_trig Nb_Sc
  1:    Sc    Sc     miss P1      96    21
  2:    Sc other     miss P1      96    21
  3: other    Sc       cr P1      96    21
  4:    Sc other     miss P1      96    21
  5: other other       cr P1      96    21
 ---                                      
140: other  resp       fa P2     104    25
141: other other       cr P2     104    25
142: other other       cr P2     104    25
143: other  resp       fa P2     104    25
144: other  resp       fa P2     104    25  
  • Note that undefined categories like
    197: P2 resp other 104 25
    have been dropped from the join result and are not counted.

Modifying the categorisation

The 2-D lookup table is easily modified. For instance, we can introduce a new category dr (duplicated responses) to cover cases where a participant has pressed the button two or more times in a row.

lut2D <- fread(
  "t1    other Sc   resp
   other cr    cr   fa
   Sc    miss  miss hit
   resp  NA    NA   dr"
)

With the modified categorisation and Sinh Nguyen's data we get

   ID Nb_trig Nb_Sc hit miss fa cr dr
1: P1      96    21   4   17 15 33  8
2: P2     104    25   6   19 17 33  6

This result can help to explain the differences between Sinh Nguyen's answer and AnilGoyals's answer as discussed in Sinh Nguyen's comment.

Related