How do I include rows from dataframe that contain certain keywords

user_16213488 0
Viewed 60

I'm analysing reddit threads for an assignment and I only want to include threads that contain certain keywords.

I have a list of keywords: keywords <- c(addict', 'addicted', 'addiction','addictive', 'afraid' ,'anxiety','anxious','cry','crying','delusion','delusional')

The dataframe has 3 columns. I want to only include rows that contain one of the keywords in the column called title.

e.g.

title created_utc
1 Anyone have a RH wallet yet? Asking for a friend 164128421
2 Ravi Menon, managing director of the Monetary Auth... 164131283
3 Different Augmented Reality(AR) NFT apps and marke... 164134123

keywordstest2<-paste0(keywords, collapse = "|") dfsub%>% filter(grepl(keywordstest2,title))

Tried this, obvs didn't work.

Does anyone know how to do this. Thanks :D

reddit dplyr filter r
4 Answers

This should work.

library(tidyverse)

dfsub %>% 
filter(grepl('addict|addicted|addiction|addictive|afraid|anxiety|anxious|cry|crying|delusion|delusional', title))
user_29693930

You can try this. I extended the example to contain 2 of the keywords

EDIT, as mentioned by Merijn in the comments adding word boundaries \\b to exclude false positives since grepl does partial matching

library(dplyr)

keywords <- c("addict", "addicted", "addiction", "addictive", "afraid", "anxiety",
"anxious", "cry", "crying", "delusion", "delusional")

df %>% filter(grepl(paste0("\\b",paste(keywords, 
  collapse="\\b|\\b"),"\\b"), df$title))
  id                                                             title
1  1         Anyone have a RH wallet yet? Asking delusion for a friend
2  3 Different Augmented Reality(AR) NFT apps and anxiety and marke...
  created_utc
1   164128421
2   164134123

Data

df <- structure(list(id = 1:3, title = c("Anyone have a RH wallet yet? Asking delusion for a friend",
"Ravi Menon managing director of the Monetary Auth... crypto", "Different Augmented Reality(AR) NFT apps and anxiety and marke..."
), created_utc = c(164128421L, 164131283L, 164134123L)), class = "data.frame", row.names = c(NA,
-3L))
user_94620950

Here is another tidyverse option. I collapse your keywords into a searchable list (e.g., addict or addicted or ...). Then, I use str_detect on title to look for any of those keywords and if so, then those rows are kept (using filter).

library(tidyverse)

df %>% 
  filter(str_detect(title, paste(keywords, collapse = "|")))

Or base R, you can filter in one line:

df[grep(paste(keywords, collapse = "|"),df$title),]

Output

  id                                                             title created_utc
1  1         Anyone have a RH wallet yet? Asking delusion for a friend   164128421
2  3 Different Augmented Reality(AR) NFT apps and anxiety and marke...   164134123

Data

df <-
 structure(
   list(
     id = 1:3,
     title = c(
       "Anyone have a RH wallet yet? Asking delusion for a friend",
       "Ravi Menon managing director of the Monetary Auth...",
       "Different Augmented Reality(AR) NFT apps and anxiety and marke..."
     ),
     created_utc = c(164128421L, 164131283L, 164134123L)
   ),
   class = "data.frame",
   row.names = c(NA,-3L)
 )

keywords <- c('addict', 'addicted', 'addiction','addictive', 'afraid' ,'anxiety','anxious','cry','crying','delusion','delusional')

For filtering on 2 columns, then you could do something like this:

df %>%
  filter(Reduce(`|`, across(
    c(title, selftext), .fns = ~ str_detect(., paste(keywords, collapse = "|"))
  )))
user_152931910

For this relative small keyword list concatenation the | is an option, but you run into issues when that string to match becomes too large. Also this far the answers given also matches "crypto" based on a keyword "cry". I adapted the df a bit to include the word "crypto".

df <- structure(list(id = 1:3, title = c("Anyone have a RH wallet yet? Asking delusion for a friend",
"Ravi Menon managing director of the Monetary crypto Auth...", "Different Augmented Reality(AR) NFT apps and anxiety and marke..."
), created_utc = c(164128421L, 164131283L, 164134123L)), class = "data.frame", row.names = c(NA,
-3L))

keywords <- c("addict", "addicted", "addiction", "addictive", "afraid", "anxiety",
"anxious", "cry", "crying", "delusion", "delusional")

library(stringr)

df %>% 
  group_by(id) %>%
  filter(any(stri_trans_tolower(stri_extract_all_words(title)[[1]]) %in% keywords))

# # A tibble: 2 x 3
# # Groups:   id [2]
#      id title                                                             created_utc
#   <int> <chr>                                                                   <int>
# 1     1 Anyone have a RH wallet yet? Asking delusion for a friend           164128421
# 2     3 Different Augmented Reality(AR) NFT apps and anxiety and marke...   164134123
user_104157490
Related