Where does R store %>% operator results while running?

user_13253740 0
Viewed 55

I'm trying to optimize long runtime operations in my R code and guessing why it takes too much time.

I'm working with large data and my code is something like this:

var <- data.table(data) %>%
    select(some_features) %>%
    setnames(new_names) %>%
    merge(blablabla) %>%
    filter(screenName %in% DEFAULT_USERS) %>% 

    # Some long runtime operations because large data
    ...... %>%
    merge(some_df) %>% data.table() %>%
    select(some_boring_features)

I imagine that R stores all variables in RAM (same as all languages)... But when %>% is used, those running operations results are not stored in variable yet and I suggest that are stored in cache memory instead of using RAM memory. Is it right?

Would it be better to instance new variables and save partial results in RAM instead of executing the whole thread?

Thanks!

ram r caching profiling performance
1 Answers

Since you are looking for which call(s) is(are) taking the most time, instancing is the route I would suggest. Certainly I always do so when debugging a sequence of calls is required.

In my view piping is more of an end which improves code readability and maintenance, not a beginning to construct and test correct logic. YMMV.

user_132880870
Related