I am trying to fit a simple linear model with OLS. I have say 4 terms and the number of data points is >> 4, so no explicit overfitting.

1) My terms or my independent variables do have a strong correlation among each other
2) They are all positively correlated with my y

My question is that on building the OLS model, I actually see the coefficient on some independent variables to be negative.

I was expecting all positive signs actually, given the positive correlation. What could cause this? What is a remedy, Non Neg Least Squares.

  • 1
    $\begingroup$ It sounds like you are suffering from multicollinearity. This happens when your independent variables are highly correlated. $\endgroup$ – Akavall Oct 10 '12 at 18:53
  • 1
    $\begingroup$ There are a lot of threads on CV already that discuss this issue. You might start by reading these: regression-coefficients-that-flip-sign-after-including-other-predictors & coefficients-change-signs, but there are others, if you search for them. You may also want to search under the multicolinearity tag. If you still have a question after reading these, come back & clarify what you still need to know. $\endgroup$ – gung - Reinstate Monica Oct 10 '12 at 18:55
  • 1
    $\begingroup$ Hi Ganesh, welcome to the site. I changed your +ve'ly to 'positively'. Many readers of this site have English as a second (or third, or fourth!) language and non-standard abbreviations may be hard for them to follow. $\endgroup$ – Peter Flom Oct 10 '12 at 19:22
  • 1
    $\begingroup$ Collinearity is not the same as correlation; you can have high correlation without problematic collinearity and you can have low correlation and high collinearity. To test for collinearity, use condition indexes, not correlation. The links provided byu @gung have details. $\endgroup$ – Peter Flom Oct 10 '12 at 19:24
  • 1
    $\begingroup$ Don't worry about being a newbie, @ganesh, almost everyone asking questions here is. But this issue has been covered many times already, in depth, on this site. Why don't you start by reading through the linked threads, searching around the site & reading a few more as your ideas become clearer, & then, if you still have any residual confusion, come back here & update your Q w/ what you still need to know. $\endgroup$ – gung - Reinstate Monica Oct 10 '12 at 19:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.