Significant value change after remove intercept in Linear model

user_2636687 0
Viewed 687

I have implemented a linear regression with intercept and without an intercept:

TotalReview ~ Number_of_files + LOC

With intercept, I get the below output where Number_of_files variable is significant:

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)   
(Intercept)     -5.279e+02  1.114e+02  -4.740  0.00515 **
LOC              7.045e-04  2.260e-03   0.312  0.76778   
Number_of_files  1.929e+00  6.026e-01   3.202  0.02395 *

Without Intercept, I get a very different output and LOC is suddenly significant:

Coefficients:
                 Estimate Std. Error t value Pr(>|t|)  
Number_of_files -0.760434   0.433852  -1.753   0.1302  
LOC              0.008528   0.003302   2.582   0.0416 *

Why does the significance of my variable change from Number_of_files to LOC after the intercept is removed?

linear-regression regression r
1 Answers

Intuitively, what a regression does, is fitting a line in the "best possible way" through a cloud of data points. A coefficient in your regression output is the slope of this line. If the slope (coefficient) is zero, then (according to the regression logic) there is no relation between the dependent variable y and the independent variable x, i.e. the coefficient will be insignificant.

When you decide to remove the intercept from the fitted line, the slopes of the lines will change in an attempt to still fit the best possible line through the cloud of the data points. What you are seeing is exactly that: the coefficients of LOC and Number_of_files have changed dramatically.

In the image below you can see the impact in the case where you only have one independent variable (say, only LOC). As you can see, the slope of the blue line (no intercept) is much steeper than the slope of the red line (with intercept).

enter image description here

On a final note, unless you have a very good reason why your model should not contain an intercept, you should just keep the intercept.

user_62564820
Related