Compute minima across list elements with NA

user_8400969 0
Viewed 60

I got a klugey solution but feel silly writing so much code for what seems simple. This goes pretty fast with lists of a few dozen MB, so I don't need to improve efficiency. But I'd still like help.

I have a large list (n elements, each one is a vector of length m). I need to get the m minimum values across all n elements (what I mean is obvious in code if this is confusing). There are NAs, in some cases with 0 complete cases and in most cases with >=1 complete case. I wrote some code that works fine but it feels like there should be a much simpler way to get here. Can you streamline this code?

Specifically, is there a way to avoid the conditional for the minimum function, and is there an apply-family function that would let me avoid the first cbind?

# make data
rawval<-replicate(10, sample(c(1:10, NA), size = 10, replace =T)
     , simplify = F)

# this seems clunky, does this function have a name?
mymin<-function(x)ifelse(all(is.na(x)), NA, min(x, na.rm =T))

# I don't see why I should need two apply family functions here
tomin<-sapply(rawval, cbind) %>%  apply(MARGIN = 1, FUN = mymin)

Apologies, I suspect this is a duplicate question :(

apply min r
3 Answers

You may use do.call and cbind the dataset and apply hablar::min_ function rowwise using apply. hablar::min_ returns NA if all the values are NA.

apply(do.call(cbind, rawval), 1, hablar::min_)

You may also use your own function if you don't want to use hablar::min_.

custom_min <- function(x) if(all(is.na(x))) NA else min(x, na.rm = TRUE)
apply(do.call(cbind, rawval), 1, custom_min)
user_39629140

What you want is mapply. It applies a function to every element of multiple lists. See its help page.

I'll suggest you a function. I'm not really sure about the sum part, but if I got it right, you only want to find the min of the rows which have a positive sum.

I benchmarked my_function against your_function and got the following results:

UPDATE: I also included in the benchmark a my_updated_function, where I simply use pmin.int. I now understand your point, if all values are NA, keep NA as the "min". I previously thought there would be negative values.

I included the pmin solution you suggested (using ifelse) and the @jay.sf solution.

rawval <- replicate(
    1000,
    sample(c(1:10, NA), size = 1000, replace =T),
    simplify = F
)

my_function <- function(values) {
    sums <- mapply(sum, values, na.rm=TRUE)
    mins <- mapply(min, values, na.rm=TRUE)
    mins[sums <= 0] <- NA
    return(mins)
}

my_updated_function <- function(values) {
    mins <- do.call(pmin.int, c(values, na.rm=TRUE))
    # if min is zero, all values are NA. NOTE: this only works like this
    # because I'm assuming numbers will always be positive integers
    # like the example you provided.
    mins[mins == 0] <- NA
    return(mins)
}

your_function <- function(values) {
    mymin<-function(x)ifelse(sum(x, na.rm=T)>0, min(x, na.rm =T), NA)
    
    # I don't see why I should need two apply family functions here
    tomin<- apply(sapply(values, cbind), MARGIN = 1, FUN = mymin)
    return(tomin)
}

pmin_function <- function(values) {
    sums <- mapply(sum, values, na.rm=TRUE)
    mins <- do.call(pmin, c(values, na.rm = TRUE))
    mins[sums <= 0] <- NA
    return(mins)
}

jay_sf_function <- function(values) {
    return(sapply(values, \(x) ifelse(!all(is.na(x)), min(x, na.rm=TRUE), NA)))
}

microbenchmark::microbenchmark(
    your_function(rawval),
    my_function(rawval),
    my_updated_function(rawval),
    pmin_function(rawval),
    jay_sf_function(rawval)
)

Unit: milliseconds
                        expr     min       lq      mean   median       uq     max neval
       your_function(rawval) 29.0871 32.77735 34.676408 34.37340 35.91040 77.6884   100
         my_function(rawval)  4.8762  5.16365  5.376355  5.37335  5.52475  7.3706   100
 my_updated_function(rawval)  2.6481  2.72655  2.872085  2.78275  2.92460  4.0724   100
       pmin_function(rawval)  5.7140  5.95945  6.268012  6.13110  6.35375  9.4198   100
     jay_sf_function(rawval)  4.8583  5.13700  6.839790  5.43480  6.45270 47.6075   100
user_139303890

Are you looking for this? This is more based on your description than your code.

rawval2
# [[1]]
# [1] 2 5 4 2 1 3
# 
# [[2]]
# [1]  5 NA  4  1  5  2
# 
# [[3]]
# [1]  1 NA  3 NA  2  1
# 
# [[4]]
# [1] NA NA NA  5 NA NA
# 
# [[5]]
# [1] NA NA NA NA NA NA

sapply(rawval2, \(x) ifelse(!all(is.na(x)), min(x, na.rm=TRUE), NA))
# [1]  1  1  1  5 NA

Data

rawval2 <- list(c(2L, 5L, 4L, 2L, 1L, 3L), c(5L, NA, 4L, 1L, 5L, 2L), c(1L, 
NA, 3L, NA, 2L, 1L), c(NA, NA, NA, 5L, NA, NA), c(NA_integer_, 
NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_
))
user_65740380
Related