# Proving an identity involving $E(e_i^2)$ in simple OLS

Once expressed the simple OLS residual $$e_i$$ as a weighted sum of the noise terms:

$$\begin{equation}e_{i}=\sum_{j}\left(\delta_{i j}-\frac{1}{n}-\left(x_{i}-\overline{x}\right) \frac{x_{j}-\overline{x}}{n s_{X}^{2}}\right) \varepsilon_{j},\end{equation}$$

where $$\delta_{i j}$$ is 1 when $$i=j$$ and 0 otherwise, I need to prove that:

$$E(e_{i}^{2})=\sigma^{2}\left(1-\frac{1}{n}-\frac{\left(x_{i}-\overline{x}\right)^{2}}{n s_{X}^{2}}\right).$$

However, I squared the first equation and then took the expected value and exploiting the fact that $$E(\varepsilon_i\varepsilon_j) = 0$$ for $$i\ne j$$, I arrive to:

$$E(e_{i}^{2})=\sigma^{2}\sum_{j}\left(\delta_{i j}-\frac{1}{n}-\left(x_{i}-\overline{x}\right) \frac{x_{j}-\overline{x}}{n s_{X}^{2}}\right) ^2.$$

What I'm doing wrong?

• Thanks, I corrected the question with: $E(\varepsilon_i\varepsilon_j) = 0.$ It was a typo, so I think $E(e_{i}^{2})=\sigma^{2}\sum_{j}\left(1-\frac{1}{n}-\left(x_{i}-\overline{x}\right) \frac{x_{j}-\overline{x}}{n s_{X}^{2}}\right) ^2$ is true. – pdb Mar 8 '19 at 10:58
• OK, now simply develop the square. Alternatively use matrix computations involving the hat matrix which you expressed. – Yves Mar 8 '19 at 11:12
• Once developed the square, things become messy, and I do not see how, from squared terms, I get to $\left(1-\frac{1}{n}-\frac{\left(x_{i}-\overline{x}\right)^{2}}{n s_{X}^{2}}\right).$ – pdb Mar 8 '19 at 13:36
• OK then go for matrices. The hat matrix $\mathbf{H}:= \mathbf{X}[\mathbf{X}^\top\mathbf{X}]^{-1}\mathbf{X}^\top$ has element $H_{ij} = [1 + (x_i - \bar{x})(x_j - \bar{x})/ s_X^2] / n$ as you wrote. We have $\mathbf{e} = [ \mathbf{I} - \mathbf{H}] \boldsymbol{\varepsilon}$ and the main point is that $\mathbf{H}$ and $\mathbf{I} - \mathbf{H}$ are orthogonal projection matrices. – Yves Mar 8 '19 at 14:20
• Ok, I computed the hat matrix and I have seen that it has the form you indicated. Then I computed $E(\sum e_i) = E(\mathbf{e'}\mathbf{e}) = E(\boldsymbol{\varepsilon}'[ \mathbf{I} - \mathbf{H}] \boldsymbol{\varepsilon})$ and saw that the terms that remain are those having form $\sigma^{2}\left(1-\frac{1}{n}-\frac{\left(x_{i}-\overline{x}\right)^{2}}{n s_{X}^{2}}\right)$, so I suppose I can express each $E(e_{i}^{2})$ in that way. So I think I messed up somewhere with the first multiplication $e_ie_i$. – pdb Mar 8 '19 at 17:03

\begin{align} E(e_{i}^{2}) &=\sigma^{2}\sum_{j}\left(\delta_{i j}-\frac{1}{n}-\left(x_{i}-\overline{x}\right) \frac{x_{j}-\overline{x}}{n s_{X}^{2}}\right) ^2 \\ &=\sigma^{2}\left(1-\frac{1}{n} -2\frac{\left(x_{i}-\overline{x}\right)^{2}}{n s_{X}^{2}} -\frac{1}{n} + \frac{n}{n^2} + 2\frac{\left(x_{i}-\overline{x}\right)\overbrace{\sum_j\left(x_{j}-\overline{x}\right)}^{0}}{n^2s_{X}^{2}} + \\ + \frac{\left(x_{i}-\overline{x}\right)^{2}\overbrace{\sum_j\left(x_{j}-\overline{x}\right)^2}^{ns_{X}^{2}}}{n^2 s_{X}^{2}s_{X}^{2}}\right) \\ &=\sigma^{2}\left(1-\frac{1}{n} - \frac{\left(x_{i}-\overline{x}\right)^{2}}{n s_{X}^{2}} \right) \end{align}