Measuring regression coefficients contribution using anova

user_11238057 0
Viewed 181

I wanted to ask about anova F-test. Can this test be used to measure coefficients contribution on dependent variable?

Anova test compares variations between and within a certain groups and in linear regression we use it to test wheater all regression coefficiets (apart from intercept) are equal to zero.

My question is if we can compare sum square of residuals for each term used in regression as follows:

library(tidyverse)


mtcars %>% 
  lm(hp ~ factor(gear) + factor(am) + qsec + wt, data = .) %>% 
  aov() %>% 
  broom::tidy() %>% 
  mutate(contribution = sumsq/sum(sumsq))

Can we intepret this summary output as: gear explains about 44 percents of variability in dependent variable? That qsec explains about 25% of variablitity? Can this approach be used in practice as e.g. If we have to choose ONLY one variable for explaining dependent variable we should choose gear in this example with only this variables?

I would like to ask if this is correct or not and most importantly why

anova statistics r
1 Answers

Yes your interpretation is correct. One way to see this is that if we take the sum of all sumsq, substract the sumsq of the Residuals and divide this quantity by the sum of sumsq, we get the R-squared of the regression (so the % of total variance explained by our model). 

mtcars %>% 
  lm(hp ~ factor(gear) + factor(am) + qsec + wt, data = .) %>% 
  aov() %>% 
  broom::tidy() %>% 
  mutate(contribution = sumsq/sum(sumsq),
         r_squared = (sum(sumsq) - sumsq[term == "Residuals"]) / sum(sumsq))

# A tibble: 5 x 8
  term            df  sumsq meansq statistic       p.value contribution r_squared
  <chr>        <dbl>  <dbl>  <dbl>     <dbl>         <dbl>        <dbl>     <dbl>
1 factor(gear)     2 64213. 32106.    33.3    0.0000000680      0.441       0.828
2 factor(am)       1   759.   759.     0.787  0.383             0.00521     0.828
3 qsec             1 36567. 36567.    37.9    0.00000165        0.251       0.828
4 wt               1 19098. 19098.    19.8    0.000144          0.131       0.828
5 Residuals       26 25090.   965.    NA     NA                 0.172       0.828

mtcars %>% 
  lm(hp ~ factor(gear) + factor(am) + qsec + wt, data = .) %>% 
  summary() %>% .["r.squared"]
$r.squared
[1] 0.8278279

The proportion of variance explained by one main effect (or interaction effect) is called "eta squared" and is an effect size measure. There are functions to compute this, for example with the lsr package:

aov_model <- aov(hp ~ (gear) + (am) + qsec + wt, data = mtcars) 
library(lsr)
etaSquared(aov_model, type = 1)

          eta.sq eta.sq.part
gear 0.440638981   0.7190454
am   0.005210947   0.0293768
qsec 0.250925660   0.5930678
wt   0.131052319   0.4321958

You could use this in practice to select the most important variables, but I would choose other models like lasso regression to do that.

user_62421320
Related