How to use R's {collapse} package to achieve a correct fgroup_by() |> ftransform() output?

user_6105259 0
Viewed 154

{collapse} is an R package that allows speedier processing times for both data manipulation and descriptive statistics. Some functionality intentionally echoes {dplyr}. For example, compare the following dplyr code versus collapse code. They are very similar in syntax and yield the same output.

library(ggplot2)
library(dplyr, warn.conflicts = FALSE)
library(collapse, warn.conflicts = FALSE)
#> collapse 1.6.5, see ?`collapse-package` or ?`collapse-documentation`
#> Note: stats::D  ->  D.expression, D.call, D.name

using_dplyr <- 
  mpg |>
  group_by(manufacturer) |>
  summarise(mean_hwy = mean(hwy))

using_collapse <- 
  mpg |>
  collapse::fgroup_by(manufacturer) |>
  collapse::fsummarise(mean_hwy = mean(hwy))

identical(using_collapse, using_dplyr)
#> [1] TRUE

But {collapse} is just faster

bench::mark(dplyr = 
              mpg |>
              group_by(manufacturer) |>
              summarise(mean_hwy = mean(hwy)),
            
            collapse = 
              mpg |>
              fgroup_by(manufacturer) |>
              fsummarise(mean_hwy = mean(hwy))) |>
  autoplot()

Created on 2021-08-26 by the reprex package (v2.0.0)

My question:
Whereas the following expressions are analogous and working:
group_by() |> summarise() = fgroup_by() |> fsummrise(),
the following analogy doesn't work:
group_by() |> mutate() |> != fgroup_by() |> ftransform(). (ftransform() is collapse's equivalent to dplyr::mutate()).

Demonstrating the problem
Let's show the same pipe in two equivalent syntaxes:

library(ggplot2)
library(dplyr, warn.conflicts = FALSE)
library(collapse, warn.conflicts = FALSE)
#> collapse 1.6.5, see ?`collapse-package` or ?`collapse-documentation`
#> Note: stats::D  ->  D.expression, D.call, D.name

is_bigger_than_mean <- function(x) {
  x > mean(x)
}

## dplyr syntax
mutate_using_dplyr <- 
  mpg |>
  select(manufacturer, hwy) |>
  group_by(manufacturer) |>
  mutate(hwy_bigger_than_mean = is_bigger_than_mean(hwy))

## collapse syntax
mutate_using_collapse <- 
  mpg |>
  fselect(manufacturer, hwy) |>
  fgroup_by(manufacturer) |>
  ftransform(hwy_bigger_than_mean = is_bigger_than_mean(hwy))

Compare the results in the column hwy_bigger_than_mean

mutate_using_dplyr  ## correct
#> # A tibble: 234 x 3
#> # Groups:   manufacturer [15]
#>    manufacturer   hwy hwy_bigger_than_mean
#>    <chr>        <int> <lgl>               
#>  1 audi            29 TRUE                
#>  2 audi            29 TRUE                
#>  3 audi            31 TRUE                
#>  4 audi            30 TRUE                
#>  5 audi            26 FALSE               
#>  6 audi            26 FALSE               
#>  7 audi            27 TRUE                
#>  8 audi            26 FALSE               
#>  9 audi            25 FALSE               
#> 10 audi            28 TRUE                
#> # ... with 224 more rows

mutate_using_collapse ## incorrect
#> # A tibble: 234 x 3
#>    manufacturer   hwy hwy_bigger_than_mean
#>  * <chr>        <int> <lgl>               
#>  1 audi            29 TRUE                
#>  2 audi            29 TRUE                
#>  3 audi            31 TRUE                
#>  4 audi            30 TRUE                
#>  5 audi            26 TRUE                
#>  6 audi            26 TRUE                
#>  7 audi            27 TRUE                
#>  8 audi            26 TRUE                
#>  9 audi            25 TRUE                
#> 10 audi            28 TRUE                
#> # ... with 224 more rows
#> 
#> Grouped by:  manufacturer  [15 | 16 (11)]

Created on 2021-08-26 by the reprex package (v2.0.0)

What causes the difference? And is it possible to achieve a correct output such that mutate_using_collapse will give the same output as mutate_using_dplyr?

note: a similar question discusses a similar issue here

dplyr r
2 Answers

As Konrad has pointed out, ftransform() is not a 1-to-1 stand-in for mutate() as it's, by design, more limited in scope, doesn't respect grouped data by default, and is mainly intended to be used with a restricted set of "fast" functions in the package.

You can use the fmean() function to achieve your goal but need to explicitly tell it to use the grouped data using the GRP() function.

Note also that it won't work with the new base R pipes, you need to use magrittr pipes for the grouping data to be available to the function call.

library(ggplot2)
library(dplyr)
library(collapse)

mpg %>%
   fselect(manufacturer, hwy) %>%
   fgroup_by(manufacturer) %>%
   ftransform(hwy_bigger_than_mean = hwy > fmean(hwy, GRP(.), TRA = "replace")) 

# A tibble: 234 x 3
   manufacturer   hwy hwy_bigger_than_mean
 * <chr>        <int> <lgl>               
 1 audi            29 TRUE                
 2 audi            29 TRUE                
 3 audi            31 TRUE                
 4 audi            30 TRUE                
 5 audi            26 FALSE               
 6 audi            26 FALSE               
 7 audi            27 TRUE                
 8 audi            26 FALSE               
 9 audi            25 FALSE               
10 audi            28 TRUE                
# ... with 224 more rows
user_28352610

As of version 1.7.0 of collapse there is now a 'fmutate' function that does computations on grouped data.

mpg %>%
   fselect(manufacturer, hwy) %>%
   fgroup_by(manufacturer) %>%
   fmutate(hwy_bigger_than_mean = hwy > fmean(hwy))
user_5359170
Related