R using dplyr::select() in a list-column workflow

Question

I have a list of large data frames and I want to subset each one, retaining only certain columns. The names of the columns I want are contained in character vectors unique to each data frame.

One way of doing this is with a list-column workflow. I would create a data frame with a data list-column holding the data frames, and a cols list-column holding the character vectors.

The real application of this will include a list of 24 large datasets, paired with a list of 24 unique character vecotrs. Here is a minimal example of this data structure to illustrate the problem:

set.seed(2346)
df <- tibble(
  col1 = sample(c(0,1), replace=T, size=10),
  col2 = sample(c(0,1), replace=T, size=10),
  col3 = sample(c(0,1), replace=T, size=10),
  col4 = sample(c(0,1), replace=T, size=10)
)

cols <- c("col1", "col3")

df_list_col <- tibble(
  data = list(df), 
  cols = list(cols)
)

df_list_col has the list-column structure, but only in a single row.

My attempted solution is to create a third list-column to hold the subsetted data frame. Thus:

df_output <- df_list_col %>% 
  mutate(subset = select(.$data, !!.$cols))

But this returns an error:

#   Error: Problem with `mutate()` input `subset`.
# x `select()` doesn't handle lists.
# ℹ Input `subset` is `select(.$data, list(c("col1", "col3")))`.

I also tried using purrr::map to apply the function:

df_output <- df_list_col %>% 
  mutate(subset = map(.$data, ~ select(.x, !!.$cols)))

But that returns a similar error. In both cases, select() is seeing the vector of column names as a list, not as vector. And I'm stumped on how to change this behavior.

Thanks in advance for any help!

user_3732271 · Answer

Both are list columns. We can extract by unlisting or extracting with [[ in select

dplyr::select(df_list_col$data[[1]], unlist(df_list_col$cols))

Or another option with !!!

select(df_list_col$data[[1]], !!! df_list_col$cols)

Or using the tidyverse syntax

library(dplyr)
library(purrr)
df_list_col %>% 
         mutate(subset = map2(data, cols, ~ .x %>% select(all_of(.y))))

-output

# A tibble: 1 x 3
#  data              cols      subset           
#                             
#1

Or with pmap

df_list_col %>%
     mutate(subset = pmap(cur_data(),  ~ select(..1, all_of(..2 ))))

R using dplyr::select() in a list-column workflow

1 Answers