When I subset a dataframe according to certain conditions, the result I get is the subsetted conditions AND NA rows.
When i use filter, it only gives me the filtered rows.
SO subsetting I may get 20 rows (10 REAL and 10 NA rows) BUT filtering, I get 10 rows (ALL Real).
Why is this?
It is a behavior specified in the documentation of ?filter
The filter() function is used to subset a data frame, retaining all rows that satisfy your conditions. To be retained, the row must produce a value of TRUE for all conditions. Note that when a condition evaluates to NA the row will be dropped, unlike base subsetting with [.
Main reason is that any comparison using operators == or > or < returns NA if the element is an NA as
NA > 5
#[1] NA
NA == 5
#[1] NA
However, if we use %in%, it returns FALSE
NA %in% 5
#[1] FALSE
So, in filter, if we need to also return the NA elements, use an | condition
df1 %>%
filter(col1 == 5|is.na(col1))