ggplot2 warning "removed x rows containing missing values" when drop = FALSE

user_12908375 0
Viewed 1161

I am creating a side-by-side bar chart with ggplot2. My code produces the correct plot when scale_x_discrete(drop = T). However, I have a level with a value of 0, and I want to include this on the x-axis. When I set scale_x_discrete(drop = F), I get the warning: Removed x rows containing missing values (geom_bar). and another category with a non-zero value is shown as zero on the plot.

Here's a reprex of my data:

library("tidyverse")

df <- data.frame(
  location = c(rep("in", 231), rep("out", 83)),
  status = c(rep("normal", 73), rep("mild", 42), rep("moderate", 20), rep("fever", 4),
             rep("normal", 70), rep("mild", 41), rep("moderate", 62), rep("fever", 2)))

df$status <- factor(df$status, levels = c("normal", "mild", "moderate", "severe", "fever"))


df %>%
  ggplot(aes(x = status,
             y = ..count../tapply(..count.., ..x.., sum)[..x..],
             fill = location)) +
  geom_bar(position = "dodge") +
  scale_y_continuous(labels = scales::percent) +
  scale_x_discrete(drop=F) +
  NULL

I have been looking at this for ages and really cannot work out the problem.

ggplot2 r
2 Answers

Can't explain the non-zero values not plotting. Here is a solution using a dplyr's group_by functions

#calculate totals and then calculate the %
df %>% group_by(status, location) %>% summarise(value=n()) %>%   
  group_by(status) %>% mutate(result=value/sum(value)) %>%.      
  ggplot(aes(x = status,
             y = result,
             fill = location)) +
  geom_col(position = "dodge") +
  scale_y_continuous(labels = scales::percent) +
  scale_x_discrete(drop=F)

Notice this now geom_col and not geom_bar.

enter image description here

user_57922440

Your code does not work because even with drop = FALSE the missing categories are still not present in ..count.. and ..x... This can be seen by plotting ..count.. and ..x...

library("tidyverse")

df <- data.frame(
  location = c(rep("in", 231), rep("out", 83)),
  status = c(rep("normal", 73), rep("mild", 42), rep("moderate", 20), rep("fever", 4),
             rep("normal", 70), rep("mild", 41), rep("moderate", 62), rep("fever", 2)))

df$status <- factor(df$status, levels = c("normal", "mild", "moderate", "severe", "fever"))

Plot ..count..

df %>%
  ggplot(aes(x = status,
             y = ..count..,
             fill = location)) +
  geom_bar(position = "dodge") +
  scale_x_discrete(drop=F)

The missing categories are not present in ..count.. which we can infer from the fact that for normal only one value shows up, i.e. ..count.. is the vector

..count.. <- c(143, 64, 19, 20, 62, 4, 2)

Plot ..x..

df %>%
  ggplot(aes(x = status,
             y = ..x..,
             fill = location)) +
  geom_bar(position = "dodge") +
  scale_x_discrete(drop=F)

As with ..count.. the missing categories are not present in ..x.. i.e. ..x.. is the vector

..x.. <- c(1, 2, 2, 3, 3, 5, 5)

Why the code does not work

As a first step I compute tapply(..count.., ..x.., sum) which gives us a vector of length 4 (total counts for non-missing status categories):

tapply(..count.., ..x.., sum)
#>   1   2   3   5 
#> 143  83  82   6

Now, extracting the elements via [..x..] results in

tapply(..count.., ..x.., sum)[..x..]
#>    1    2    2    3    3 <NA> <NA> 
#>  143   83   83   82   82   NA   NA

or

..count.. / tapply(..count.., ..x.., sum)[..x..]
#>      1      2      2      3      3   <NA>   <NA> 
#> 1.0000 0.7711 0.2289 0.2439 0.7561     NA     NA

Hence your code results in two missings for the last two categories, which explains the warning Removed 2 rows containing missing values (geom_bar). The reason is that with ..x.. <- c(1, 2, 2, 3, 3, 5, 5) we are trying to extract two times the 5th element from the length 4 vector tapply(..count.., ..x.., sum) and therefore get NAs back.

In case of drop=TRUE everything works fine because in that case ..x.. <- c(1, 2, 2, 3, 3, 4, 4) while ..count.. is the same.

Solution

The issue can be solved by converting ..x.. to a character vector. In that case we extract elements by names:

library("tidyverse")

df <- data.frame(
  location = c(rep("in", 231), rep("out", 83)),
  status = c(rep("normal", 73), rep("mild", 42), rep("moderate", 20), rep("fever", 4),
             rep("normal", 70), rep("mild", 41), rep("moderate", 62), rep("fever", 2)))

df$status <- factor(df$status, levels = c("normal", "mild", "moderate", "severe", "fever"))

# Convert ..x.. to character
df %>%
  ggplot(aes(x = status,
             y = ..count.. / tapply(..count.., ..x.., sum)[as.character(..x..)],
             fill = location)) +
  geom_bar(position = "dodge") +
  scale_x_discrete(drop=F)

Created on 2020-03-23 by the reprex package (v0.3.0)

user_129938610
Related