Negative coefficients in regression for positive coefficient of correlation

user_16543033 0
Viewed 451

I am building a multiple linear regression model using python. I found the correlation coefficients between all the independent variables and the dependent variable. They were all greater than 0.5. However, the equation given by the regression model has a few negative coefficients. Why is that?

coefficients correlation linear-regression machine-learning python
1 Answers

The situation you describe is not impossible. The key is to analyse the correlation between your independent variables. If there is a strong correlation between two of them, it is possible that one of these two has a negative coefficient in your linear regression.

For instance, take the following example where you want to predict y using x1and x2 as independant variables :

  • Let's say that y is deterministic and follows y = x1 + 2 * x2.
  • Let's say that x2is also deterministic and that x2 = 0.1 * x1

Then, your linear regression has an infinite number of possiblity, you could say that y = 1.2 * x1 + 0 * x2 as well as y = 0 * x1 + 12 * x2 or y = 2 * x1 - 8 * x2. In the latter case, although there is a positive correlation between yand x2, you have a negative coefficient in your linear regression. There is no error, nothing wrong.

That is why you should not jump to conclusions and say that because there is a positive slope in your linear regression model, that necessarily means that there is a positive correlation between the independent and the predicted variable. And of course, that you cannot infer any causation from your results.

I hope that answers the question.

user_82442470
Related