How to define column that doesn't have NAs in sequence of certain rows?

Viewed 60

Here is some sample data:

    df1 <- read.table(text = "Date  Client1 Client2 Client3
                  01.01.2019    0   0   2
                  01.02.2019    0   0   3
                  01.03.2019    0   0   4
                  01.04.2019    0   0   4
                  01.05.2019    0   0   4
                  01.06.2019    1   0   4
                  01.07.2019    0   0   0
                  01.08.2019    0   0   1
                  01.09.2019    0   0   0
                  01.10.2019    0   3   0
                  01.11.2019    0   0   2
                  01.12.2019    2   0   0
                  01.01.2020    3   4   3
                  01.02.2020    4   0   3
                  01.03.2020    5   0   0
                  01.04.2020    5   0   0
                  ", header = TRUE)
df1[df1 == 0] <- NA

And the question is how to find a logical index for every column that contains a sequence of 5 or more rows that doesn't contains NAs.

Client1 TRUE
Client2 FALSE
Client3 TRUE
3 Answers

I'd use the rle() function to count run lengths of !is.na(). For example, using your definition of df1:

df2 <- data.frame(Name = character(3), Group = character(3))

for (i in 1:3) {
  runs <- rle(!is.na(df1[, i + 1]))
  good <- which(runs$values == TRUE)
  runs <- runs$lengths
  n <- length(runs)
  df2$Group[i] <- if (n %in% good) "Stable"
                  else if (max(runs[good]) >= 5) "Was_Stable"
                  else "Not_Stable"
  df2$Name[i] <- names(df1)[i + 1]
}

You can use sapply to iterate over columns and check with any :

sapply(df1[-1], function(x) any(with(rle(!is.na(x)), values & lengths >= 5)))

# Client1 Client2 Client3 
#   TRUE   FALSE    TRUE 

Similar to user2554330, then you can use rle like this:

# add a column with no NAs as an example
df1 <- cbind(df1, dummy = 1:NROW(df1)) 

# find columns with five or more NAs in a row
is_num <- vapply(df1, is.numeric, TRUE) # assume we only look at numerics?
res <- setNames(rep(TRUE, NCOL(df1)), colnames(df1))
res[is_num] <- vapply(df1[is_num], function(x){
  o <- rle(!is.na(x))
  any(o$lengths[o$values] > 4)
}, TRUE)
res
#R> Date Client1 Client2 Client3   dummy 
#R> TRUE    TRUE   FALSE    TRUE    TRUE

I would expect this to be quite fast. If you do not care about the other columns, then you can do:

is_num <- vapply(df1, is.numeric, TRUE)
vapply(df1[is_num], function(x){
  o <- rle(!is.na(x))
  any(o$lengths[o$values] > 4)
}, TRUE)
#R> Client1 Client2 Client3 
#R>    TRUE   FALSE    TRUE

I realise after the fact that this is only a minor twist to the answer by Ronak Shah. Combining his approach with my solution yields this:

vapply(df1[vapply(df1, is.numeric, TRUE)], function(x)
  with(rle(!is.na(x)), any(lengths[values] > 4)), TRUE)
#R> Client1 Client2 Client3 
#R>    TRUE   FALSE    TRUE
Related