mixing unnest_longer and unnest_wider

MarkusN 0
Viewed 65

I'm (once more) stuck with flattening nested lists.

I have this tibble with some list-columns (originating from a JSON format).

    library(tidyr)
    library(dplyr)
    df = tibble(id = c(1, 2, 3),
            branch = list(NULL, list(colA = 'abc', colB = 'mno'),
                          list(list(colA = 'def', colB = 'uvw'),
                               list(colA = 'ghi', colB = 'xyz'))))

I want to unnest_wider column 'branch'. That works with rows 1 and 2:

df %>% 
  slice(1:2) %>% 
  unnest_wider(branch)

However, row 3 consists of a list of lists which I have to unnest_longer first:

bind_rows(
  df %>% slice(1,2),
  df %>% slice(3) %>% unnest_longer(branch)) %>% 
  unnest_wider(branch)

above code gives the desired output, but I'm looking for a generic solution like:

If an element of column 'branch' is of type 'unnamed list' (indicating that there is a list of lists) then unnest_longer. Afterwards apply unnest_wider to the whole column 'branch'

Any help appreciated!

tidyr purrr r json
4 Answers

A little bit convoluted but here's a possible solution:

  1. Iterate through the rows of your df
  2. Determine if it's a named list by checking names(df$branch[[index]])
  3. If unnamed --> slice + unnest; if named --> slice
  4. Finally, unnest_wider()
library(tidyr)
library(dplyr)
library(purrr)

map_df(1:nrow(df), function(x) {
  if (is.null(names(df$branch[[x]]))) {
    df %>% slice(x) %>% unnest_longer(branch)
  } else {
    df %>% slice(x)
  }
}) %>% 
  unnest_wider(branch)

Which returns:

# A tibble: 4 × 3
     id colA  colB 
  <dbl> <chr> <chr>
1     1 NA    NA   
2     2 abc   mno  
3     3 def   uvw  
4     3 ghi   xyz
userUzpS0 
library(tidyverse)
df <- tibble(
  id = c(1, 2, 3),
  branch = list(
    NULL, list(colA = "abc", colB = "mno"),
    list(
      list(colA = "def", colB = "uvw"),
      list(colA = "ghi", colB = "xyz")
    )
  )
)

unnester <- function(x, grp) {
  if (grp) {
    x <- x |> unnest_longer(branch)
  }
  unnest_wider(x, branch)
}

df |>
  rowwise() |>
  mutate(grp = length(names(unlist(branch))) > 2) |>
  ungroup() |>
  split(~grp) |>
  imap_dfr(~ unnester(.x, .y)) |>
  select(-grp)
Nir Graham0

The following approach first modifies the list so that all leafs are located at the same list level, after which we can unnest all rows as needed:

library(tidyr)
library(purrr)
library(dplyr)

mutate(df, branch = map(
  .x = branch, 
  .f = ~if(is.list(.x[[1]])) .x else list(.x)
)) |>
  unnest_longer(branch) |>
  unnest_wider(branch)

#> # A tibble: 4 × 3
#>      id colA  colB 
#>   <dbl> <chr> <chr>
#> 1     1 <NA>  <NA> 
#> 2     2 abc   mno  
#> 3     3 def   uvw  
#> 4     3 ghi   xyz
Joris C.0

First convert the leaves to data frames and then unnest it.

library(dplyr)
library(tidyr)

leaf2df <- function(x) {
  if (length(names(x))) as.data.frame(x)
  else if (is.list(x)) lapply(x, leaf2df)
}

df %>%
  rowwise %>%
  mutate(branch = list(bind_rows(leaf2df(branch)))) %>%
  ungroup %>%
  unnest(branch, keep_empty = TRUE)

giving:

# A tibble: 4 × 3
     id colA  colB 
  <dbl> <chr> <chr>
1     1 <NA>  <NA> 
2     2 abc   mno  
3     3 def   uvw  
4     3 ghi   xyz

Because leaf2df is recursive as long as all leaves in any row have the same parent it should continue to work. For example, below we have made the list in the last row one level deeper and it still works.

df <- tibble(id = c(1, 2, 3),
    branch = list(NULL, list(colA = 'abc', colB = 'mno'),
                  list(list(list(colA = 'def', colB = 'uvw'),
                       (list(colA = 'ghi', colB = 'xyz'))))))
G. Grothendieck0
Related