Cohen's d effect size for multiple simple effect comparisons with more than 2 levels (following an interaction)

user_7737143 0
Viewed 319

I'm trying to find an easy way to compute Cohen's d (standardised mean difference) for multiple simple effect comparisons following an interaction.

In this case, I have one factor with 2 levels and a second factor with 3 levels. I'd compute a 2x2 ANOVA and find an interaction. Then I'd want to follow up with the specific t-test comparisons and report an effect size.

If there is a package or a simple function or an easy way to do this, please share!

First, make some data:

df1 <- data.frame(cond1 = rep(0:1, 500),
                  cond2 = sample(0:2, 1000, replace = TRUE),
                  dv = rnorm(1000, 2, 1)
                  )
#fit the model
model <- lm(dv ~ cond1*cond2, df1)

test pairwise comparisons for the interaction (which isn't sig. here but pretend that it is)

emm <- emmeans::emmeans(model, pairwise ~ cond1|cond2)
#can do cond1|cond2 or cond2|cond1, both work

This seems like it should work, but I can't figure out why I get this error message:

emmeans::eff_size(emm, sigma = sigma(model), edf = df.residual(model))
#Error in update.default(object, tran = NULL) : need an object with call component

This works:

summary(psych::cohen.d.by(df1 ~ cond1 + cond2))

but this does not work if I wanted to get the pairwise comparisons stratified the other way:

summary(psych::cohen.d.by(df1 ~ cond2 + cond1))
#Error in `.rowNamesDF<-`(x, value = value) : invalid 'row.names' length

If I only had one condition variable, I would use rstatix:: However, as far as I know, this package and function does not allow to input more than 1 grouping variable.

rstatix::cohens_d(df1, dv ~ cond)

Any other suggestions?

What I'm looking for is the standardised mean difference for each comparison in a list for each comparison. I know it's a lot of comparisons, but it's a common procedure in social science and there should be a function made to do this.

anova r
1 Answers

There is nothing wrong with your code. It runs as given:

set.seed(123)

df1 <- data.frame(cond1 = rep(0:1, 500),
                  cond2 = sample(0:2, 1000, replace = TRUE),
                  dv = rnorm(1000, 2, 1)
)

model <- lm(dv ~ cond1*cond2, df1)

emm <- emmeans::emmeans(model, pairwise ~ cond1|cond2)

emmeans::eff_size(emm, sigma = sigma(model), edf = df.residual(model))
## Since 'object' is a list, we are using the contrasts already present.
## cond2 = 1.02:
##  contrast effect.size     SE  df lower.CL upper.CL
##  (0 - 1)       0.0245 0.0633 996  -0.0996    0.149
## 
## sigma used for effect sizes: 1.006 
## Confidence level used: 0.95

Created on 2021-08-23 by the reprex package (v2.0.0)

So you must have done some other manipulations (e.g., that replace model with something else) before you ran eff_size()

I do suggest, however, that you set up cond2 as a factor in your model. You are treating it like a numerical predictor, and that's why it gets reduced to its average (1.02 in this case) rather than having separate effect sizes for each level.

user_39615660
Related