This question comes from MånsT's answer of question

The least squares estimators of $β_1$,$β_2$,… are not affected by shifting. The reason is that these are the slopes of the fitting surface - how much the surface changes if you change $x_1$,$x_2$,… one unit. This does not depend on location. (The estimator of $β_0$, however, does.)

Such question first arised when I centered the predictor value. Can someone give a rigorous proof maybe by matrix language? And how would it change intercept term? Some related question:
- Show that $\hat{\beta}_0 = \bar{y}$ for OLS when the columns of $\mathbf{X}$ are centered
- Why does the y-intercept of a linear model disappear when I standardize variables?
- Prove Estimated Regression Coefficients are the same with or without an intercept term

  • 1
    $\begingroup$ One rigorous solution follows from the final remark in my answer to a related question at stats.stackexchange.com/a/108862/919: "The constant term will be the difference between the mean of $y$ and the mean values predicted from the estimates, $X\hat\beta.$" The answer itself shows that $\hat\beta$ depends only on the covariance matrix of all the variables and that obviously does not change when the predictors are shifted. That should also make clear that the conclusion follows only when the constant vector lies in the span of the columns of the design matrix. $\endgroup$ – whuber Apr 28 '19 at 16:05
  • $\begingroup$ @whuber Thanks! Can you provide a geometric interpretation for this property? I think the answer above are geometric interpretation,but I can't figure out $\endgroup$ – Spaceship222 Apr 29 '19 at 2:27
  • $\begingroup$ @whuber By the way, in your posted answer, if $\alpha$ is intercept, I think the system of linear equations should be $$ X' X \hat{b}=X' y $$ where $\hat{b} = (\hat{\alpha},\hat{\beta})$. Then after Gaussian elimination, we get $$ C \hat{\beta}=\left(\operatorname{Cov}\left(X_{i}, y\right)\right)' $$ Am I wrong? $\endgroup$ – Spaceship222 Apr 29 '19 at 2:28
  • $\begingroup$ A geometric interpretation is that the fitted value at the barycenter of the regressors is the barycenter of the response variable(s). $\endgroup$ – whuber Apr 29 '19 at 13:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.