Execute dplyr operation only if column exists

user_1655567 0
Viewed 15870

Drawing on the discussion on conditional dplyr evaluation I would like conditionally execute a step in pipeline depending on whether the reference column exists in the passed data frame.

Example

The results generated by 1) and 2) should be identical.

Existing column

# 1)
mtcars %>% 
  filter(am == 1) %>%
  filter(cyl == 4)

# 2)
mtcars %>%
  filter(am == 1) %>%
  {
    if("cyl" %in% names(.)) filter(cyl == 4) else .
  }

Unavailable column

# 1)
mtcars %>% 
  filter(am == 1)

# 2)    
mtcars %>%
  filter(am == 1) %>%
  {
    if("absent_column" %in% names(.)) filter(absent_column == 4) else .
  }

Problem

For the available column the passed object does not correspond to the initial data frame. The original code returns the error message:

Error in filter(cyl == 4) : object 'cyl' not found

I have tried alternative syntax (with no luck):

>> mtcars %>%
...   filter(am == 1) %>%
...   {
...     if("cyl" %in% names(.)) filter(.$cyl == 4) else .
...   }
 Show Traceback

 Rerun with Debug
 Error in UseMethod("filter_") : 
  no applicable method for 'filter_' applied to an object of class "logical"

Follow-up

I wanted to expand this question that would account for the evaluation on the right-hand side of the == in filter call. For instance the syntax below attempts to filter on the first available value. mtcars %>%

filter({
    if ("does_not_ex" %in% names(.))
      does_not_ex
    else
      NULL
  } == {
    if ("does_not_ex" %in% names(.))
      unique(.[['does_not_ex']])
    else
      NULL
  })

Expectedly, the call evaluates to an error message:

Error in filter_impl(.data, quo) : Result must have length 32, not 0

When applied to existing column:

mtcars %>%
  filter({
    if ("mpg" %in% names(.))
      mpg
    else
      NULL
  } == {
    if ("mpg" %in% names(.))
      unique(.[['mpg']])
    else
      NULL
  })

It works with a warning message:

  mpg cyl disp  hp drat   wt  qsec vs am gear carb
1  21   6  160 110  3.9 2.62 16.46  0  1    4    4

Warning message: In { : longer object length is not a multiple of shorter object length

Follow-up question

Is there a neat way of expending the existing syntax in order to get conditional evaluation on the right-hand side of the filter call, ideally staying within dplyr workflow?

dplyr dataframe lazy-evaluation r function
6 Answers

With across() in dplyr > 1.0.0 you can now use any_of when filtering. Compare original with all columns:

mtcars %>% 
  filter(am == 1) %>% 
  filter(cyl == 4)

With cyl removed, it throws an error:

mtcars %>% 
  select(!cyl) %>% 
  filter(am == 1) %>% 
  filter(cyl == 4)

Using any_of (note you have to write "cyl" and not cyl):

mtcars %>% 
  select(!cyl) %>% 
  filter(am == 1) %>% 
  filter(across(any_of("cyl"), ~.x == 4))
#N.B. this is equivalent to just filtering by `am == 1`.
user_132585250

I know I'm late to the party, but here's an answer somewhat more in line with what you were originally thinking:

mtcars %>%
  filter(am == 1) %>%
  {
    if("cyl" %in% names(.)) filter(., cyl == 4) else .
  }

Basically, you were missing the . in filter. Note this is because the pipeline doesn't add . to filter(expr) since it is in an expression surrounded by {}.

user_50160120

Avoid this trap:

On a busy day, one might do like the following: 

library(dplyr)
df <- data.frame(A = 1:3, B = letters[1:3], stringsAsFactors = F)
> df %>% mutate( C = ifelse("D" %in% colnames(.), D, B)) 
# Notice the values on "C" colum. No error thrown, but the logic and result is wrong
  A B C
1 1 a a
2 2 b a
3 3 c a

Why? Because "D" %in% colnames(.) returns only one value of TRUE or FALSE, and therefore ifelse operates only once. Then the value is broadcasted to the whole column!

Correct way:

> df %>% mutate( C = if("D" %in% colnames(.)) D else B)
  A B C
1 1 a a
2 2 b b
3 3 c c
user_5888670

This code does the trick and is pretty flexible. The ^ and $ are regex used to perform an exact match. 

mtcars %>% 
  set_names(names(.) %>% 
              str_replace("am","1") %>% 
              str_replace("^cyl$","2") %>% 
              str_replace("Doesn't Exist","3")
              )
user_63366700

Edit: Unfortunately, this was too good to be true

I might be a bit late to the party. But is

mtcars %>% 
 filter(am == 1) %>%
 try(filter(absent_column== 4))

a solution?

user_108672410
Related