0
$\begingroup$

What assumptions does one make about the residuals in linear regression? Say if we didn't have an intercept, would the sum of the residuals be guaranteed to equal zero? if we used a different loss function (e.g., absolute loss), would the sum be guaranteed to equal zero?

$\endgroup$
  • 1
    $\begingroup$ No assumption on residuals. We have the definition of residuals which is $Y-\hat Y$. $\endgroup$ – user158565 Nov 29 '18 at 4:02
2
$\begingroup$

What assumptions does one make about the residuals in linear regression?

For estimates to be unbiased and consistent, residual must

  1. have mean zero, i.e. $E[\varepsilon_i|X]=0$ (conditional mean zero assumption)
  2. must be independent and identically distributed; i.e. $cov(\varepsilon_i,\varepsilon_j)=0$ (i.i.d. assumption).

For the estimates being efficient, one also requires

  1. Constant variance; i.e. $Var(\varepsilon_i|X)=\sigma_\varepsilon$ (homoscedasticity)

  2. Residuals are normally distributed; i.e. $\varepsilon_i$~$N(0,\sigma_\varepsilon)$ (normality assumption).

Generally, assumptions 1-3 are the important ones with assumption 4 being asymptotically fulfilled for large samples (say more than 50).

Say if we didn't have an intercept, would the sum of the residuals be guaranteed to equal zero?

Yes, see Comments below.

If we used a different loss function (e.g., absolute loss), would the sum be guaranteed to equal zero?

It of course depends on the loss function. But if you don't want to violate the conditional mean zero assumption, residuals should better add up to 0.

$\endgroup$
  • $\begingroup$ "No, just think of the model $y=\varepsilon_i$ without any other variables"...no sure how $y=\varepsilon_i$ related to no-interception, shouldn't it be look like $y_i=a.x_i$ (where with interception it is $y_i=a.x_i+b$) $\endgroup$ – william007 Nov 30 '18 at 12:20
  • $\begingroup$ I changed the example in my response. Hope this makes sense now to you. $\endgroup$ – Tom Pape Dec 1 '18 at 14:31
  • $\begingroup$ You seem to equate residuals with error terms. This is wrong. $\endgroup$ – Michael M Dec 1 '18 at 17:53
  • $\begingroup$ Yes, @MichaelM is right- residuals will always have conditional mean 0 when estimated by OLS (that is the moment condition that is exploited). $\endgroup$ – ChinG Dec 1 '18 at 18:03
  • $\begingroup$ Ah right, indeed I did. Thanks for pointing this out Michael & Chin $\endgroup$ – Tom Pape Dec 2 '18 at 10:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.