Accessing results from a list column when some elements are NA

user_2829961 0
Viewed 40

Problem: List column contains a few missing values

Data

Consider the following tibble that contains the results of 2 model fits:

> Model_fits
# A tibble: 4 x 4
    cyl               data model1    model2   
  <dbl> <list<tibble[,2]>> <list>    <list>   
1     2            [5 x 2] <dbl [1]> <dbl [1]>
2     4           [11 x 2] <lm>      <lm>     
3     6            [7 x 2] <lm>      <dbl [1]>
4     8           [14 x 2] <lm>      <lm>

The data for cyl==2 was missing in this example. Therefore, model1 contains NA_real_ in the first row. Similarly, model2 contains NA_real_ in rows 1 and 3.

Extracting model results

I want to extract the results of model fit using broom::glance. But it does not work due to the missing values:

> Model_fits %>% 
+   mutate(summary_res = map(model1, broom::glance))
Error: Problem with `mutate()` input `summary_res`.
x No glance method for objects of class numeric
i Input `summary_res` is `map(model1, broom::glance)`.

Attempt at solution

So, I try to use purrr::possibly, but that does not work either:

> Model_fits %>% 
+   mutate(summary_res1 = map(model1, ~ possibly(broom::glance(.x),
+                                              otherwise = NA_real_)))
Error: Problem with `mutate()` input `summary_res1`.
x No glance method for objects of class numeric
i Input `summary_res1` is `map(model1, ~possibly(broom::glance(.x), otherwise = NA_real_))`.

Expected outcome

I want to get the broom::glance results for all non-missing values and NA_real_ for all missing values. Please guide me how can I get these results?

Code for creating Model_fits

Please note that I created the following as a reproducible example. But this is not my original data/model results.

library(tidyverse)

new_data <- tibble(mpg = rep(NA_real_, 5),
       cyl = rep(2, 5),
       disp = rep(NA_real_, 5))

mtcars2 <- mtcars %>% 
  dplyr::select(mpg, cyl, disp)

mt <- bind_rows(mtcars2, 
                new_data)
  
model_res_list <- map(mtcars2 %>% group_split(cyl), ~lm(mpg ~ disp, data = .x))

lizt <- list(NA_real_, model_res_list[[1]], model_res_list[[2]], model_res_list[[3]])

lizt2 <- list(NA_real_, model_res_list[[1]], NA_real_, model_res_list[[3]])


Model_fits <- mt %>% 
  group_nest(cyl) %>% 
  mutate(model1 = lizt,
         model2 = lizt2)
purrr tidyverse r try-catch
2 Answers

One more thing you could do about this is using tryCatch function, So that you define in case of an error occurring what would be the output of your function. In this case it will not bring the execution of the function to a halt.

Model_fits %>%
  mutate(mod01 = map(model1, ~ tryCatch(glance(.x), 
                                        error = function(cond) {
                                          NA_real_
                                        }))) %>%
  unnest(mod01)

# A tibble: 4 x 17
    cyl         data model1  model2 mod01 r.squared adj.r.squared sigma statistic  p.value    df
  <dbl> <list<tibbl> <list>  <list> <dbl>     <dbl>         <dbl> <dbl>     <dbl>    <dbl> <dbl>
1     2      [5 x 2] <dbl [~ <dbl ~    NA   NA             NA     NA      NA      NA          NA
2     4     [11 x 2] <lm>    <lm>      NA    0.648          0.609  2.82   16.6     0.00278     1
3     6      [7 x 2] <lm>    <dbl ~    NA    0.0106        -0.187  1.58    0.0537  0.826       1
4     8     [14 x 2] <lm>    <lm>      NA    0.270          0.209  2.28    4.44    0.0568      1
# ... with 6 more variables: logLik <dbl>, AIC <dbl>, BIC <dbl>, deviance <dbl>,
#   df.residual <int>, nobs <int>

If we want to use possibly or safely instead of tryCatch we should first write a custom function that wraps glance in general and before applying on our data set:

poss_glance <- possibly(glance, otherwise = NA_real_)

Model_fits %>%
  mutate(mod01 = map(model1, ~ poss_glance(.x))) %>%
  unnest(mod01)

# A tibble: 4 x 17
    cyl         data model1  model2 mod01 r.squared adj.r.squared sigma statistic  p.value    df
  <dbl> <list<tibbl> <list>  <list> <dbl>     <dbl>         <dbl> <dbl>     <dbl>    <dbl> <dbl>
1     2      [5 x 2] <dbl [~ <dbl ~    NA   NA             NA     NA      NA      NA          NA
2     4     [11 x 2] <lm>    <lm>      NA    0.648          0.609  2.82   16.6     0.00278     1
3     6      [7 x 2] <lm>    <dbl ~    NA    0.0106        -0.187  1.58    0.0537  0.826       1
4     8     [14 x 2] <lm>    <lm>      NA    0.270          0.209  2.28    4.44    0.0568      1
# ... with 6 more variables: logLik <dbl>, AIC <dbl>, BIC <dbl>, deviance <dbl>,
#   df.residual <int>, nobs <int>

Or even we could use safely in place of possibly so that our function returns an enhanced output in this case NA_real_:

safe_glance <- safely(glance, otherwise = NA_real_)

Model_fits %>%
  mutate(mod01 = map(model1, ~ safe_glance(.x)))
user_143145200

You can just check if the value is NA in a wrapper function that you pass to map.

Model_fits %>% 
   mutate(summary_res = map(model1, function(x) if (length(x) == 1 && is.na(x)) NA_real_ else  broom::glance(x)))
user_120001720
Related