0
$\begingroup$

One of the assumptions of linear regression is that the residual mean is zero. As far as I can tell though, the residual mean is always zero i.e. it is not an assumption, it is a fact.

The formula for calculating the least squares line means that the sum of all residuals is zero and the mean of all residuals is zero (even if the line is not actually linear).

Can someoen explain if/why this is not the case? Would appreciate an "english" response rather that jumping straight into the maths please!

$\endgroup$
  • 5
    $\begingroup$ The linear regression assumptions concern the model errors, not the model residuals. The model residuals estimate the (unknown) model errors. Since the model errors are assumed to have mean zero, then the model residuals "mimic" that and have an average of zero (hence a sum of zero). A good starting point for you would be to understand the difference between the model errors and the model residuals. $\endgroup$ – Isabella Ghement Nov 18 '18 at 23:54
  • $\begingroup$ See this link for more details on the difference between the model errors and model residuals: stats.stackexchange.com/questions/133389/…. $\endgroup$ – Isabella Ghement Nov 19 '18 at 15:29
  • 1
    $\begingroup$ Echoing @Isabella, it might be worthwhile pointing out that you use "residual mean" in two distinct ways. The residual mean assumption refers to the expectations of random variables in a model. The residual mean result--namely, "is always zero"--refers to a statistic (ie, a function of the data) generated by a certain kind of procedure (namely, ordinary least squares that includes a constant term). Clearing this semantic confusion ought to suffice to improve one's understanding. $\endgroup$ – whuber Nov 19 '18 at 15:37
1
$\begingroup$

I think the above comments are quite clear. You have a true model first:

$$ y_i = x_i \beta + e_i $$

You assume the mean of the random variable, $e_i$, is zero. When you use OLS or some other methods to fit the model, you have already acknowledged this assumption. For example, OLS is minimizing the sum of square of residuals, which is kind of assuming that the error term tends to be small and, hence, the residual should be close to zero in some sense. Suppose the mean of residual is not zero and you don't have an intercept in this model, OLS might not make much sense as the errors might not be close to zero. Note you always think of a true model first, and then you discuss the estimation.

It's also worth mentioning that the mean of residual is not necessarily zero. It will be zero only when you have constant in the model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.