Reversed signs of test statistic using lm() vs. t.test()

Viewed 183

Why do test statistic values have reversed signs when using the lm() vs. t.test() for a single 2 factor variable for a linear morel? E.g. below:

sim_data = data.frame(
  groups = c(rep("A", n / 2), rep("B", n / 2)),
  values = rep(0, n))

sim_data$values = rnorm(n, mean = 42, sd = 3.5) 
summary(lm(values ~ groups, data = sim_data))
t.test(values ~ groups, data = sim_data, var.equal = TRUE)

Both t.test() and lm() produce the same test statistic but with different signs: 0.02 and -0.02. Is it because e.g. here, R is using Group A as reference level with lm() but using Group B as reference level with t.test()? Please enlighten me.

2 Answers

Since the help for t.test does not discuss this let us write out the various possibilities so we can empirically determine how it is acting (or look at the source code links in the comments).

The short answer is that we can make lm give the same result if we use contr.SAS contrasts in lm (as in #4 in the Comparison section) which is the same as the default contr.treatment contrasts except that the last level is used as the reference level instead of the first.

default method

Without the formula method t.test can be written as t.test(x, y, ...) and the t statistic resulting from that is the mean of x minus the mean of y all divided by the standard error so the sign of the statistic is determined by the order in which x and y are passed as arguments.

# the standard error does not depend on which level is reference level
# so just get it in most convenient way
out <- t.test(values ~ groups, sim_data, var.equal = TRUE)
out$statistic
       t 
1.118583 

stderr <- out$stderr
x <- with(sim_data, values[groups == "A"])
y <- with(sim_data, values[groups == "B"])

(mean(x) - mean(y)) / stderr
## [1] 1.118583

(mean(y) - mean(x)) / stderr
## [1] -1.118583

formula method

If we use the formula interface with a character column for groups it will effectively convert groups to a factor with the alphabetically first group as the first argument and the alphabetically second as the second argument.

# A is passed as the first argument to t.test.default
gA <- factor(sim_data$groups, levels = c("A", "B"))
t.test(values ~ gA, sim_data, var.equal = TRUE)$statistic
##        t 
## 1.118583 

# B is passed as the first argument to t.test.default
gB <- factor(sim_data$groups, levels = c("B", "A"))
t.test(values ~ gB, sim_data, var.equal = TRUE)$statistic
##         t 
## -1.118583 

lm

With lm evidently the first level is effectively passed to t.test.default as the second argument and is referred to as the base or reference level.

# A is reference level
coef(summary(lm(values ~ gA, data = sim_data)))
##              Estimate Std. Error   t value     Pr(>|t|)
## (Intercept) 42.533345  0.8827383 48.183414 1.750263e-28
## gAB         -1.396417  1.2483805 -1.118583 2.728234e-01

# B is reference level
coef(summary(lm(values ~ gB, data = sim_data)))
##              Estimate Std. Error   t value     Pr(>|t|)
## (Intercept) 41.136929  0.8827383 46.601500 4.407622e-28
## gBA          1.396417  1.2483805  1.118583 2.728234e-01

or we can control the reference level, also known as the base level, in lm using the contrasts argument (or using relevel as in the other answer):

# A is reference level (default is first level)
coef(summary(lm(values ~ groups, data = sim_data)))
##              Estimate Std. Error   t value     Pr(>|t|)
## (Intercept) 42.533345  0.8827383 48.183414 1.750263e-28
## groupsB     -1.396417  1.2483805 -1.118583 2.728234e-01

# same - groups2 in output is groups with the first level, A, as base 
coef(summary(lm(values ~ groups, data = sim_data, 
  contrasts = list(groups = contr.treatment(2, base = 1)))))
##              Estimate Std. Error   t value     Pr(>|t|)
## (Intercept) 42.533345  0.8827383 48.183414 1.750263e-28
## groups2     -1.396417  1.2483805 -1.118583 2.728234e-01

# change reference level - groups1 in output is groups with 2nd level, B, as base
coef(summary(lm(values ~ groups, data = sim_data, 
  contrasts = list(groups = contr.treatment(2, base = 2)))))
##              Estimate Std. Error   t value     Pr(>|t|)
## (Intercept) 41.136929  0.8827383 46.601500 4.407622e-28
## groups1      1.396417  1.2483805  1.118583 2.728234e-01

# contr.SAS is like contr.treatment except last level is reference
coef(summary(lm(values ~ groups, data = sim_data, 
  contrasts = list(groups = contr.SAS))))
##              Estimate Std. Error   t value     Pr(>|t|)
## (Intercept) 41.136929  0.8827383 46.601500 4.407622e-28
## groups1      1.396417  1.2483805  1.118583 2.728234e-01

Comparison

It may be easier to compare if we write the similar expressions together. These all give the same t statistic.

attach(sim_data)   
out <- t.test(values ~ groups, var.equal = TRUE)
stderr <- out$stderr
x <- values[groups == "A"]
y <- values[groups == "B"]

# 1
(mean(x) - mean(y)) / stderr
## 1.118583 

# 2
t.test(x, y, var.equal = TRUE)$statistic
##        t 
## 1.118583 

# 3
coef(summary(lm(values ~ groups, 
  contrasts = list(groups = contr.treatment(2, base = 2)))))
##              Estimate Std. Error   t value     Pr(>|t|)
## (Intercept) 41.136929  0.8827383 46.601500 4.407622e-28
## groups1      1.396417  1.2483805  1.118583 2.728234e-01

# 4 - contr.SAS is like contr.treatment except reference level 
#  defaults to last, rather than first, level
coef(summary(lm(values ~ groups, 
  contrasts = list(groups = contr.SAS))))
##              Estimate Std. Error   t value     Pr(>|t|)
## (Intercept) 41.136929  0.8827383 46.601500 4.407622e-28
## groups1      1.396417  1.2483805  1.118583 2.728234e-01

and these all give the same t statistic.

# 1
(mean(y) - mean(x)) / stderr
## [1] -1.118583

# 2
t.test(y, x, var.equal = TRUE)$statistic
##         t 
## -1.118583 

# 3
coef(summary(lm(values ~ groups, 
  contrasts = list(groups = contr.treatment(2, base = 1)))))
##              Estimate Std. Error   t value     Pr(>|t|)
## (Intercept) 42.533345  0.8827383 48.183414 1.750263e-28
## groups2     -1.396417  1.2483805 -1.118583 2.728234e-01

# 4
coef(summary(lm(values ~ groups)))
##              Estimate Std. Error   t value     Pr(>|t|)
## (Intercept) 42.533345  0.8827383 48.183414 1.750263e-28
## groupsB     -1.396417  1.2483805 -1.118583 2.728234e-01

Note

To be reproducible the input needs to define n and to issue a set.seed due to the use of random numbers. We used:

set.seed(123)
n <- 30
sim_data <- data.frame(
  groups = c(rep("A", n / 2), rep("B", n / 2)),
  values = rnorm(n, mean = 42, sd = 3.5))

you can set a reference group

sim_data$groups <- relevel(factor(sim_data$groups), ref = "B")
Related