Why is dplyr's filter not working with white spaces in simple filter?

user_13695873 0
Viewed 637

I'm new to learning R, and I'm trying to explore a dataset provided by the R for Data Science online community for movies - https://github.com/rfordatascience/tidytuesday/blob/master/data/2018/2018-10-23/movie_profit.csv.

As I'm going through and learning more about the filter function of dplyr, I noticed that I do not get any results when I search for a value within the "distributor" column with a white space in it - like "Walt Disney" in the example below. Searching for values that do not have a white space works just fine, like "Universal".

Also, I've tried with other columns in the dataset, like "movies" where I filter for a specific movie that has white-spaces in it; however, when I do that I run into no issues so I'm a bit puzzled. 

library(tidyverse)

movies <- read_csv(url("https://github.com/rfordatascience/tidytuesday/raw/master/data/2018/2018-10-23/movie_profit.csv")) 

test <- "20th Century Fox"

movies %>%
  filter(movie == "Dawn of the Planet of the Apes") %>%
  View()

In short, I'd love to know the why behind this so that I know how to handle this if it comes up again in any future datasets that I explore. The code that is giving me trouble is below. I want to plot the genre distribution of movies from the distributor "Walt Disney". 


movies <- read_csv(url("https://github.com/rfordatascience/tidytuesday/raw/master/data/2018/2018-10-23/movie_profit.csv")) 

test <- "20th Century Fox"

movies %>%
  filter(distributor == "Walt Disney") %>%
  ggplot(aes(x = genre)) +
  geom_bar()
dplyr r
3 Answers

There seems to a mismatch in the string

all.equal("Walt Disney", "Walt Disney" )
#[1] "1 string mismatch"

If we check the values

unique(movies$distributor)[3]
#[1] "Walt Disney"

charToRaw(unique(movies$distributor)[3])
#[1] 57 61 6c 74 c2 a0 44 69 73 6e 65 79
charToRaw("Wald Disney")
#[1] 57 61 6c 64 20 44 69 73 6e 65 79

There is some difference triggered the mismatch

It is better to copy from the value in 'distributor

library(dplyr)
library(ggplot2)
library(stringr)
movies %>%
    filter(str_detect(distributor, "Walt\\s+Disney")) %>%
   count(genre) %>%
   ggplot(aes(x = genre, y = n)) +
       geom_col()

-output

enter image description here

user_37322710

The following uses agrepl for an approximate string match and it gives the graph in akrun's answer.

movies %>% 
  filter(agrepl("Walt Disney", distributor)) %>%
  ggplot(aes(x = genre)) +
  geom_bar()
user_82454060

As mentioned in other posts, you have special characters instead of normal spaces in your tables. You can replace them with regular spaces and your code should work as normal, without having to manually copy the strings. I have added trimws to remove any trailing whitespace. Note that this also removes other special characters. 

library(tidyverse)

movies <- read_csv(url("https://github.com/rfordatascience/tidytuesday/raw/master/data/2018/2018-10-23/movie_profit.csv")) 

# this line replaces non-alphanumeric characters with a space and removes any trailing whitespace at the end. 
movies$distributor <- trimws(gsub("[^[:alnum:]]", " ", movies$distributor))
movies %>%
  filter(distributor == "Walt Disney") %>%
  ggplot(aes(x = genre)) +
  geom_bar()
user_42207880
Related