What assumptions does one make about the residuals in linear regression? Say if we didn't have an intercept, would the sum of the residuals be guaranteed to equal zero? if we used a different loss function (e.g., absolute loss), would the sum be guaranteed to equal zero?

• No assumption on residuals. We have the definition of residuals which is $Y-\hat Y$. – user158565 Nov 29 '18 at 4:02

What assumptions does one make about the residuals in linear regression?

For estimates to be unbiased and consistent, residual must

1. have mean zero, i.e. $$E[\varepsilon_i|X]=0$$ (conditional mean zero assumption)
2. must be independent and identically distributed; i.e. $$cov(\varepsilon_i,\varepsilon_j)=0$$ (i.i.d. assumption).

For the estimates being efficient, one also requires

1. Constant variance; i.e. $$Var(\varepsilon_i|X)=\sigma_\varepsilon$$ (homoscedasticity)

2. Residuals are normally distributed; i.e. $$\varepsilon_i$$~$$N(0,\sigma_\varepsilon)$$ (normality assumption).

Generally, assumptions 1-3 are the important ones with assumption 4 being asymptotically fulfilled for large samples (say more than 50).

Say if we didn't have an intercept, would the sum of the residuals be guaranteed to equal zero?

• "No, just think of the model $y=\varepsilon_i$ without any other variables"...no sure how $y=\varepsilon_i$ related to no-interception, shouldn't it be look like $y_i=a.x_i$ (where with interception it is $y_i=a.x_i+b$) – william007 Nov 30 '18 at 12:20