0
$\begingroup$

Suppose you have a regression model

$\tilde{y}$ = $X\beta$ + $\varepsilon$,

where

$\tilde{y}$ = $y$ - $\bar{y}$

and $X$ contains a constant.

If you estimate the model by OLS, does the estimated intercept has to be always zero because you demeaned $y$? If true, does it make sense to include a constant at all?

$\endgroup$
1

1 Answer 1

1
$\begingroup$

The intercept is the predicted response when all the predictors are zero, a point that may or not may not occur within the range of the dataset.

Shifting the response so that it has mean zero doesn't by the same token shift the predictors each to have mean zero.

Otherwise put, you seem to be thinking that subtracting the mean response shifts the prediction surface so that it goes through the origin at which all variables are zero, but changing one variable's level has no effect on that of other variables.

This is perhaps easiest to see with a scatter plot of one response and one predictor. Subtracting the mean response means that the regression line goes through $(0,$ mean of $x)$ but the intercept is not also 0 unless exceptionally the regression line is flat or the mean of $x$ is 0 for other reasons.

I won't supply sketches because the main point is that you should do so.

Note. I won't willingly use the word demean with this meaning.

$\endgroup$
3
  • $\begingroup$ +1. This is why we usually center $y$ and all the explanatory variables, too. In that case an intercept term is superfluous. $\endgroup$
    – whuber
    Commented May 9, 2022 at 15:40
  • $\begingroup$ after demeaning & standartization (w.r.t variance) - still intercept can matter as collinearity to some categorical features/covariates $\endgroup$
    – JeeyCi
    Commented Dec 22, 2023 at 13:15
  • $\begingroup$ "The intercept is the mean value of the response if the predictor is zero." but be carefull: means can differ even in one model $\endgroup$
    – JeeyCi
    Commented Dec 22, 2023 at 13:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.