Consider a linear regression model $$ Y=X\beta + \epsilon, $$ where $Y\in R^n$, $X = (x_1,...,x_n)^T\in R^{n\times p}$ are i.i.d. $p$-dimensional observations, $\beta\in R^p$, and $\epsilon = (\epsilon_1,...,\epsilon_n)\in R^n$ are i.i.d. Let $\hat\beta$ be the estimator of $\beta$. Then the residuals are $\hat\epsilon = (\hat\epsilon_1,...,\hat\epsilon_n)= Y - X\hat\beta = X(\beta-\hat\beta) + \epsilon$. We consider both $X$ and $\epsilon$ are random. Is it correct that $$ E(\hat\epsilon_1) = \cdots = E(\hat\epsilon_n) = E\left(\frac{1}{n}\sum_{i=1}^n \hat\epsilon_i \right)? $$


1 Answer 1


Yes, if the expectations exist (I suppose if they don't exist they might still be regarded as equally non-existent)

The $\hat\epsilon_i$ are exchangeable (the subscript order is arbitrary) so all the $E[\hat \epsilon_i]$ are the same if they exist. The expectation of the average is the average of the expectations by the linearity of expectation, so it's also the same.

In addition, if the model has an intercept then these expectations are all zero, since the sum of residuals is identically zero by the definition of $\hat\beta$.

[Of course, the conditional expectations given $x$ may well not be equal]


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.