Is there an elegant way to show a progress bar during the computation of a grouped data.frame in R?

user_10043323 0
Viewed 987

I am applying a complex function to a grouped data.frame. For simplicity reasons, here, this function will be treated as the colSums function. Let's assume we have the following data.frame:

df <- data.frame(A=runif(600000,0,1),
                 B=rep(c("group1", "group2","group3","group4","group5","group6"), 100000))

Further I want to execute a dplyr chain:

df <- df %>%
    group_by(.data$B) %>%
    summarize(colSums(across()))

During this calculation, I'd like to have a Progress bar showing the remaining time. For example:

[=========================>] 100%

I know there are solutions in dplyr, but as far as I know they are superseded. Thus, I want to use the package Progress. This Progress bar is based on a tick update during a for loop. I am wondering, if this is possible for this dplyr chain. So far, I couldnt come up with a solution. Any ideas?

dplyr grouping dataframe r progress-bar
1 Answers

Here is a solution that uses the progress package. You have to initialize the progress bar and tell it the number of groups/ticks. Then in your custom computation, you increment the bar.

library(tidyverse)
library(progress)

df <- data.frame(A=runif(600000,0,1),
                 B=rep(c("group1", "group2","group3","group4","group5","group6"), 100000)) %>% 
    group_by(B)

my_slow_function <- function(col){
    pb$tick()
    Sys.sleep(0.5)
    sum(col)
}

num_ticks <- n_groups(df)
pb <- progress_bar$new(format = "[:bar] :current/:total (:percent) elapsed :elapsed eta :eta",
                       total = num_ticks)


df %>% 
    summarize(output = my_slow_function(A))
user_127443230
Related