Filter rows based on dynamic pattern

Question

I have speech data in in a dataframe dfin column Orthographic:

df <- data.frame(
  Orthographic = c("this is it at least probably",
                   "well not probably it's not intuitive",
                   "sure no it's I mean it's very intuitive",
                   "I don't mean to be rude but it's anything but you know",
                   "well okay maybe"),
  Repeat = c(NA, "probably", "it's,intuitive", "I,mean,it's", NA),
  Repeat_pattern = c(NA, "\b(probably)\b", "\b(it's|intuitive)\b", "\b(I,mean|it's)\b", 
                     NA))

I want to filter rows based on a dynamic pattern, namely the occurrence of no, never, not as words OR n't before any of the words listed in column Repeat. However, using the pattern \b(no|never|not)\b|n't\b\s together with the alternation patterns in column Repeat_pattern, I get this error:

df %>%
   filter(grepl(paste0("\b(no|never|not)\b|n't\b\s", Repeat_pattern), Orthographic))
                             Orthographic         Repeat         Repeat_pattern
1    well not probably it's not intuitive       probably       \b(probably)\b
2 sure no it's I mean it's very intuitive it's,intuitive \b(it's|intuitive)\b
Warning message:
In grepl(paste0("\b(no|never|not)\b|n't\b\s", Repeat_pattern),  :
  argument 'pattern' has length > 1 and only the first element will be used

I don't know why "only the first element will be used" as the two pattern components seem to connect well:

paste0("\b(no|never|not)\b|n't\b\s", df$Repeat_pattern)
[1] "\b(no|never|not)\b|n't\b\sNA"                     "\b(no|never|not)\b|n't\b\s\b(probably)\b"      
[3] "\b(no|never|not)\b|n't\b\s\b(it's|intuitive)\b" "\b(no|never|not)\b|n't\b\s\b(I,mean|it's)\b"   
[5] "\b(no|never|not)\b|n't\b\sNA"

The expected output is this:

2                   well not probably it's not intuitive       probably       \b(probably)\b
3                sure no it's I mean it's very intuitive it's,intuitive \b(it's|intuitive)\b
4 I don't mean to be rude but it's anything but you know    I,mean,it's    \b(I,mean|it's)\b

user_3832970 · Answer

It looks like a vectorization issue here, you need to use stringr::str_detect here rather than grepl.

Also, you did not group the negative word alternatives well, all of them must reside in a single group and your n't is now obligatory in a string.

Alse, NA values are coerced to text and added to the regex patterns, while it seems you want to discard the items where Repeat_pattern is NA.

You can fix your code by using

df %>%
    filter(ifelse(is.na(Repeat_pattern), FALSE, str_detect(Orthographic, paste0("(?:\bno|\bnever|\bnot|n't)\b.*", Repeat_pattern))))

Output:

                                            Orthographic         Repeat         Repeat_pattern
1                   well not probably it's not intuitive       probably       \b(probably)\b
2                sure no it's I mean it's very intuitive it's,intuitive \b(it's|intuitive)\b
3 I don't mean to be rude but it's anything but you know    I,mean,it's    \b(I|mean|it's)\b

I also think the last pattern must be \b(I|mean|it's)\b, not \b(I,mean|it's)\b.

If there can only be whitespace between the "no" words and the word from Repeat column, replace .* with \s+ in my pattern. I used .*\b to make sure there is a match anywhere to the right of the "no" words.

Filter rows based on dynamic pattern

1 Answers