Group_by not functioning correctly when using filter and str_detect

user_15353735 0
Viewed 72

Group_by doesn't seem to be functioning while i'm using filter. This is a simply reproduction of my data:

data = my_data_raw_quest 

user_id     question          dv
1            Allergies?        na     
1            food choice       left
2            Allergies?        yes, I hate gluten  
2            food choice       left        
3            Allergies?        allergic to soy 
3            food choice       left                   
4            Allergies?        na
4            food choice       left             
5            Allergies?        na
5            food choice       left            
6            Allergies?        Soy 
6            food choice       right          
7            Allergies?        na
7            food choice       right

when i run this code, it seems like group_by did not function correctly.

my_data_raw_quest_2 <- 
  my_data_raw_quest %>%
  dplyr::group_by(user_id) %>%
  filter(add = TRUE, is.na(dv)|
    !str_detect(dv, "(G|g)luten"))

This results in the following data set, when Instead I would like ALL responses from users that answered "gluten" to any question removed. Notice that line 3 is removed but not line 4.

user_id     question           dv
1            Allergies?        na     
1            food choice       left
2            food choice       left        
3            Allergies?        allergic to soy 
3            food choice       left                   
4            Allergies?        na
4            food choice       left             
5            Allergies?        na
5            food choice       left            
6            Allergies?        Soy 
6            food choice       right          
7            Allergies?        na
7            food choice       right

This is the table im trying to achieve

user_id     question           dv
1            Allergies?        na     
1            food choice       left
3            Allergies?        allergic to soy 
3            food choice       left                   
4            Allergies?        na
4            food choice       left             
5            Allergies?        na
5            food choice       left            
6            Allergies?        Soy 
6            food choice       right          
7            Allergies?        na
7            food choice       right

This code represents a better representative of all the data I have, I aim to keep ALL the columns. Al though in this code below there is no example of an answer containing gluten.

structure(list(session_id = c(53039, 53039, 53039, 53039, 53039, 
53039, 53047, 53047, 53047, 53047, 53047, 53047, 53050, 53050, 
53050, 53050, 53050, 53050, 53052, 53052, 53052, 53052, 53052, 
53052, 53054, 53054, 53054, 53054, 53054, 53054, 53055, 53055, 
53055, 53055, 53055, 53055, 53056, 53056, 53056, 53056), project_id = c(495, 
495, 495, 495, 495, 495, 495, 495, 495, 495, 495, 495, 495, 495, 
495, 495, 495, 495, 495, 495, 495, 495, 495, 495, 495, 495, 495, 
495, 495, 495, 495, 495, 495, 495, 495, 495, 495, 495, 495, 495
), quest_name = c("Sociodemographic", "Sociodemographic", "Sociodemographic", 
"Sociodemographic", "Sociodemographic", "Sociodemographic", "Sociodemographic", 
"Sociodemographic", "Sociodemographic", "Sociodemographic", "Sociodemographic", 
"Sociodemographic", "Sociodemographic", "Sociodemographic", "Sociodemographic", 
"Sociodemographic", "Sociodemographic", "Sociodemographic", "Sociodemographic", 
"Sociodemographic", "Sociodemographic", "Sociodemographic", "Sociodemographic", 
"Sociodemographic", "Sociodemographic", "Sociodemographic", "Sociodemographic", 
"Sociodemographic", "Sociodemographic", "Sociodemographic", "Sociodemographic", 
"Sociodemographic", "Sociodemographic", "Sociodemographic", "Sociodemographic", 
"Sociodemographic", "Sociodemographic", "Sociodemographic", "Sociodemographic", 
"Sociodemographic"), quest_id = c(2189, 2189, 2189, 2189, 2189, 
2189, 2189, 2189, 2189, 2189, 2189, 2189, 2189, 2189, 2189, 2189, 
2189, 2189, 2189, 2189, 2189, 2189, 2189, 2189, 2189, 2189, 2189, 
2189, 2189, 2189, 2189, 2189, 2189, 2189, 2189, 2189, 2189, 2189, 
2189, 2189), user_id = c(46942, 46942, 46942, 46942, 46942, 46942, 
46946, 46946, 46946, 46946, 46946, 46946, 46947, 46947, 46947, 
46947, 46947, 46947, 46949, 46949, 46949, 46949, 46949, 46949, 
46950, 46950, 46950, 46950, 46950, 46950, 46951, 46951, 46951, 
46951, 46951, 46951, 46952, 46952, 46952, 46952), user_sex = c("male", 
"male", "male", "male", "male", "male", "male", "male", "male", 
"male", "male", "male", "male", "male", "male", "male", "male", 
"male", "male", "male", "male", "male", "male", "male", "male", 
"male", "male", "male", "male", "male", "male", "male", "male", 
"male", "male", "male", "male", "male", "male", "male"), user_status = c("test", 
"test", "test", "test", "test", "test", "guest", "guest", "guest", 
"guest", "guest", "guest", "guest", "guest", "guest", "guest", 
"guest", "guest", "guest", "guest", "guest", "guest", "guest", 
"guest", "registered", "registered", "registered", "registered", 
"registered", "registered", "guest", "guest", "guest", "guest", 
"guest", "guest", "guest", "guest", "guest", "guest"), user_age = c(23, 
23, 23, 23, 23, 23, 21, 21, 21, 21, 21, 21, 22, 22, 22, 22, 22, 
22, 21, 21, 21, 21, 21, 21, 58.9, 58.9, 58.9, 58.9, 58.9, 58.9, 
21, 21, 21, 21, 21, 21, 22, 22, 22, 22), q_name = c("vegan", 
"religious", "religious influence", "food allergies", "Other", 
"Education", "vegan", "religious", "religious influence", "food allergies", 
"Other", "Education", "vegan", "religious", "religious influence", 
"food allergies", "Other", "Education", "vegan", "religious", 
"religious influence", "food allergies", "Other", "Education", 
"vegan", "religious", "religious influence", "food allergies", 
"Other", "Education", "vegan", "religious", "religious influence", 
"food allergies", "Other", "Education", "vegan", "religious", 
"religious influence", "food allergies"), q_id = c(92827394, 
92827395, 92827396, 92827397, 92831398, 92832133, 92827394, 92827395, 
92827396, 92827397, 92831398, 92832133, 92827394, 92827395, 92827396, 
92827397, 92831398, 92832133, 92827394, 92827395, 92827396, 92827397, 
92831398, 92832133, 92827394, 92827395, 92827396, 92827397, 92831398, 
92832133, 92827394, 92827395, 92827396, 92827397, 92831398, 92832133, 
92827394, 92827395, 92827396, 92827397), order = c(1, 2, 3, 4, 
5, 6, 1, 2, 3, 4, 5, 6, 1, 2, 3, 4, 5, 6, 1, 2, 3, 4, 5, 6, 1, 
2, 3, 4, 5, 6, 1, 2, 3, 4, 5, 6, 1, 2, 3, 4), dv = c("1", "Athetiesm", 
"none", "none", "none", "Undergraduate", "5", "Nope, Atheist", 
"None", "No, i am normal", "Money", "Undergraduate", "3", "Nope, Atheist", 
"None", "No", "No", "Postgraduate (masters)", "1", "Christianity", 
"None", "No", "Cost", "Postgraduate (masters)", "2", "Nope, Atheist", 
"none", "none", "less processed, cook fresh", "Undergraduate", 
"1", "Christianity", "None", "No", "Cost", "Postgraduate (masters)", 
"6", "Nope, Atheist", NA, NA), starttime = structure(c(1607440590, 
1607440590, 1607440590, 1607440590, 1607440590, 1607440590, 1607441663, 
1607441663, 1607441663, 1607441663, 1607441663, 1607441663, 1607441637, 
1607441637, 1607441637, 1607441637, 1607441637, 1607441637, 1607442744, 
1607442744, 1607442744, 1607442744, 1607442744, 1607442744, 1607442919, 
1607442919, 1607442919, 1607442919, 1607442919, 1607442919, 1607442998, 
1607442998, 1607442998, 1607442998, 1607442998, 1607442998, 1607443123, 
1607443123, 1607443123, 1607443123), class = c("POSIXct", "POSIXt"
), tzone = "UTC"), endtime = structure(c(1607440643, 1607440643, 
1607440643, 1607440643, 1607440643, 1607440643, 1607441839, 1607441839, 
1607441839, 1607441839, 1607441839, 1607441839, 1607441714, 1607441714, 
1607441714, 1607441714, 1607441714, 1607441714, 1607442819, 1607442819, 
1607442819, 1607442819, 1607442819, 1607442819, 1607443041, 1607443041, 
1607443041, 1607443041, 1607443041, 1607443041, 1607443020, 1607443020, 
1607443020, 1607443020, 1607443020, 1607443020, 1607443148, 1607443148, 
1607443148, 1607443148), class = c("POSIXct", "POSIXt"), tzone = "UTC")), row.names = c(NA, 
40L), class = "data.frame")
stringr dplyr r
2 Answers

Edit 2: Updated with semi-join

After some thought and with your example data, I think the following should work. Note I tested it on the word "none" as I couldn't find gluten in the sample provided, so obviously change the pattern accordingly for your needs.

my_data_raw_quest %>% group_by(user_id) %>% 
# Select user_id and dv, then nest into list columns
  select(user_id,dv) %>% 
  summarise( dv = list(dv)) %>% 
# Find occurences of none in dv
mutate(none = map(dv, str_detect, pattern = "none"),
# filter out using a logical vector, replace NAs
none = !map_lgl(none, any),
none = replace_na(none,FALSE)
) %>% 
  filter(none) %>% 
# semi-join on user_id to return selected users only
    semi_join(y = ., x = my_data_raw_quest, by = "user_id")

Edit 1: Nesting / Unnesting

One way could be to nest the columns after grouping by user id, then map over a filtering sequence and unnesting again:

my_data_raw_quest %>% group_by(user_id) %>% 
# Nest into list columns
summarise(question = list(question), dv = list(dv)) %>% 
# Find occurences of gluten
mutate(gluten = map(dv, str_detect, pattern = "(G|g)luten"),
# filter out using a logical vector
gluten = !map_lgl(gluten, any)) %>% 
  filter(gluten) %>% select(-gluten) %>% 
# unnest
  unnest(c(question,dv))
user_153697220

Try the following:

library(tidyr)

my_data_raw_quest %>% 
  pivot_wider(names_from = question, values_from = dv) %>% # Reshape to wide
  filter(str_detect(`Allergies?`, "(G|g)luten", negate = TRUE) | str_detect(`food choice`, "(G|g)luten", negate = TRUE)) %>% # Filter
  pivot_longer(c(`Allergies?`, `food choice`), names_to = "question", values_to = "dv") # Reshape back to original shape

By the way, I don't know what you were looking to achieve with filter(add = TRUE, ...). add is not a valid argument for filter().

user_52216260
Related