Performing t-tests on many rows of a df when x and mu are specified and appending the result as a column

user_4643907 0
Viewed 58

I want to perform a t-test like this one and capture only the p-value:

x <- c(24.2,25.3,24.3,24.4,24.3,25)
y <- 24.4

> t.test(x, mu=y)

One Sample t-test

data:  x
t = 0.99023, df = 5, p-value = 0.3675
alternative hypothesis: true mean is not equal to 24.4
95 percent confidence interval:
 24.10741 25.05926
sample estimates:
mean of x 
 24.58333

So I adapted another function I found on here to handle the extra argument for mu and return only the p (in my real-world example, the tryCatch is very important); the goal is to send the zone columns in as the data and the target column in as the mean to test against:

t_test <- function(x, y) {
   tryCatch(
     round(t.test(x=x, mu=as.numeric(y))$p.value, 3),
     error=function(e) NA)
}

This works like it is supposed to:

> t_test(x, y)
[1] 0.367532

But I am running into all kinds of problems when I try to perform a similar exercise on a data frame, where I have several thousand of these to run. First, here is a sample df from dput:

df <- structure(list(target = c(24.4, 24.4, 24.4, 24.4, 24.4, 24.4, 
24.4, 24.4, 24.4, 24.4), zone1 = c(23.5, 23.3, 23.5, 23.7, 23.8, 
23.7, 23.6, 23.6, 23.8, 23.7), zone2 = c(24.3, 24, 24.1, 24.4, 
24.7, 24.6, 24.5, 24.6, 24.7, 24.7), zone3 = c(24.8, 24.8, 24.5, 
24.3, 24.1, 23.8, 24.2, 24.2, 24.2, 23.8), zone4 = c(24.5, 24.5, 
24.4, 24.5, 24.4, 24.3, 24.4, 24.9, 24.5, 24.5), zone5 = c(24.3, 
24.4, 24.5, 24.5, 24.5, 24.4, 24.6, 24.6, 24.8, 24.9), zone6 = c(23.1, 
23, 23.1, 23.1, 23, 22.8, 22.6, 22.7, 23.1, 23.2)), row.names = c(NA, 
-10L), class = c("tbl_df", "tbl", "data.frame"))

Here is my attempt and the error. I think maybe I am supposed to vectorize, but all my attempts to do that (and to try =apply= outside of a mutate) are failing in strange ways (including one that works fine when the =df= is small but produces only 0.00-0.02 when =df= has many rows).

> df %>% mutate(result = mapply(t_test, zone1:zone6, target))
# A tibble: 10 x 8
   target zone1 zone2 zone3 zone4 zone5 zone6 result
    <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <lgl> 
 1   24.4  23.5  24.3  24.8  24.5  24.3  23.1 NA    
 2   24.4  23.3  24    24.8  24.5  24.4  23   NA    
 3   24.4  23.5  24.1  24.5  24.4  24.5  23.1 NA    
 4   24.4  23.7  24.4  24.3  24.5  24.5  23.1 NA    
 5   24.4  23.8  24.7  24.1  24.4  24.5  23   NA    
 6   24.4  23.7  24.6  23.8  24.3  24.4  22.8 NA    
 7   24.4  23.6  24.5  24.2  24.4  24.6  22.6 NA    
 8   24.4  23.6  24.6  24.2  24.9  24.6  22.7 NA    
 9   24.4  23.8  24.7  24.2  24.5  24.8  23.1 NA    
10   24.4  23.7  24.7  23.8  24.5  24.9  23.2 NA    
Warning messages:
1: Problem with `mutate()` input `result`.
i numerical expression has 10 elements: only the first used
i Input `result` is `mapply(t_test, zone1:zone6, target)`. 
2: Problem with `mutate()` input `result`.
i numerical expression has 10 elements: only the first used
i Input `result` is `mapply(t_test, zone1:zone6, target)`.

Thanks for your insights. The solution does not have to use mutate, mapply, or apply... I am open to any solutions with multiple steps where I can end up tacking another column onto my df with the p value.

dplyr apply r
2 Answers

You need to match up your inputs to what is expected as arguments to t_test.

x = vector of values
y = vector of a single value

mapply expects a list of each of your x values, and one or multiple y values in a vector. As it stands, by passing zone1:zone6 you have 6 sets of 10 values rather than the 10 sets of 6 values you want to use. If you transpose, you can make it work:

mapply(t_test, x=data.frame(t(df[-1])), y=df[[1]])
#   X1    X2    X3    X4    X5    X6    X7    X8    X9   X10 
#0.284 0.228 0.171 0.230 0.265 0.142 0.240 0.411 0.447 0.371

Or use apply to convert to a numeric matrix, and compare all the values except the first value in each row, to the first value.

apply(df, 1, function(x) t_test(x[-1], x[1]) )
#[1] 0.284 0.228 0.171 0.230 0.265 0.142 0.240 0.411 0.447 0.371
user_4968030

In dplyr, you can use rowwise -

library(dplyr)

df %>%
  rowwise() %>%
  mutate(result = t_test(c_across(zone1:zone6), target))

#   target zone1 zone2 zone3 zone4 zone5 zone6 result
#    <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>  <dbl>
# 1   24.4  23.5  24.3  24.8  24.5  24.3  23.1  0.284
# 2   24.4  23.3  24    24.8  24.5  24.4  23    0.228
# 3   24.4  23.5  24.1  24.5  24.4  24.5  23.1  0.171
# 4   24.4  23.7  24.4  24.3  24.5  24.5  23.1  0.23 
# 5   24.4  23.8  24.7  24.1  24.4  24.5  23    0.265
# 6   24.4  23.7  24.6  23.8  24.3  24.4  22.8  0.142
# 7   24.4  23.6  24.5  24.2  24.4  24.6  22.6  0.24 
# 8   24.4  23.6  24.6  24.2  24.9  24.6  22.7  0.411
# 9   24.4  23.8  24.7  24.2  24.5  24.8  23.1  0.447
#10   24.4  23.7  24.7  23.8  24.5  24.9  23.2  0.371
user_39629140
Related