I have performed a regression in SAS and extracted the estimated correlation matrix of the parameters, which includes the intercept. One of my variable parameters has a strong correlation to the intercept parameter. I have not seen this before and want to be sure I understand what it means (if anything). Can anyone provide an intuitive explanation of what it means for a variable parameter estimate to be correlated to the intercept parameter estimate?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ If you're interested in not having that correlation in your estimates, you might consider mean-correcting your $x$'s (in some situations that makes sense to do, sometimes it doesn't). Beware, though - it changes the meaning of the intercept. $\endgroup$– Glen_bSep 21, 2013 at 1:33
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
In simple terms, imagine you fix one parameter, say the intercept, and estimate the slope. The question is: as you vary the fixed parameter, will your slope estimate change? It will to some degree, and the strength/direction of the effect is the correlation of the parameters.
Suppose you have a simple linear regression with a positive slope in the first quadrant (x and y are positive). If you begin to move the intercept up the y axis, then your slope will have to correspondingly decrease in order to pass through the data. In this case, the intercept and slope are negatively correlated. In any particular setting, the strength of the correlation will depend on the nature of the data (location, spread, etc.)
$\begingroup$
$\endgroup$
Just in case, the correlation refers to the estimated parameters, and springs from the fact that they are derived using the same data. It does not imply a correlation between the unknown parameters being estimated, which being constants (in the frequentist approach), cannot have "correlation". In that sense, it does not have an "intuitive" explanation because it is a consequence of the estimation procedure itself, and does not reflect some aspect of the real-world phenomenon under study.
answered Sep 19, 2013 at 16:02