How to estimate the mean of the 10% upper and lower values over multiple categories with dplyr?

user_7529776 0
Viewed 44

Suppose you have this data.frame in R

set.seed(15)
df <-  data.frame(cat = rep(c("a", "b"), each = 50), 
                  x = c(runif(50, 0, 1), runif(50, 1, 2)))

I want to estimate the mean of the 10% upper and lower values in each category.

I can do it using base functions like this

dfa <- df[df$cat=="a",]
dfb <- df[df$cat=="b",]

mean(dfa[dfa$x >= quantile(dfa$x, 0.9),"x"])
# [1] 0.9537632
mean(dfa[dfa$x <= quantile(dfa$x, 0.1),"x"])
# [1] 0.07959845

mean(dfb[dfb$x >= quantile(dfb$x, 0.9),"x"])
# [1] 1.963775
mean(dfb[dfb$x <= quantile(dfb$x, 0.1),"x"])
# [1] 1.092218

However, I can't figure it out how to implement this using dplyr or purrr.

Thanks for the help.

purrr dplyr r
3 Answers

We could do this in a group by approach using cut and quantile as breaks

library(dplyr)
df %>% 
    group_by(cat) %>%
    mutate(grp = cut(x, breaks = c(-Inf, quantile(x,
        probs = c(0.1, 0.9)), Inf))) %>% 
    group_by(grp, .add = TRUE) %>%
    summarise(x = mean(x, na.rm = TRUE), .groups = 'drop_last') %>%
    slice(-2)

-ouptut

# A tibble: 4 x 3
# Groups:   cat [2]
  cat   grp                x
  <chr> <fct>          <dbl>
1 a     (-Inf,0.0813] 0.0183
2 a     (0.853, Inf]  0.955 
3 b     (-Inf,1.21]   1.07  
4 b     (1.93, Inf]   1.95
user_37322710

Here's a way you can use cut() to help partitaion your data into groups and then take the mean

df %>% 
  group_by(cat) %>% 
  mutate(part=cut(x, c(-Inf, quantile(x, c(.1, .9)), Inf), labels=c("low","center","high"))) %>% 
  filter(part!="center") %>% 
  group_by(cat, part) %>% 
  summarize(mean(x))

which returns everything in a nice tibble

  cat   part  `mean(x)`
  <chr> <fct>     <dbl>
1 a     low      0.0796
2 a     high     0.954 
3 b     low      1.09  
4 b     high     1.96

To make it a bit cleaner, you can factor out the splitting to a helper function

split_quantile <- function(x , p=c(.1, .9))  {
  cut(x, c(-Inf, quantile(x, c(.1, .9)), Inf), labels=c("low","center","high"))
}
df %>% 
  group_by(cat) %>% 
  mutate(part = split_quantile(x)) %>% 
  filter(part != "center") %>% 
  group_by(cat, part) %>% 
  summarize(mean(x))
user_23720640

A variant of @MrFlick's answer - you can use cut_number and slice:

df %>% 
  group_by(cat) %>% 
  mutate(part = cut_number(x, n = 10)) %>% 
  group_by(cat, part) %>% 
  summarise(mean(x)) %>% 
  slice(1, n())
user_27999410
Related