# Where does the linear regression assumption that the errors are uncorrelated enter into the proof of Gauss Markov and that Least Squares is BLUE?

I often see that the "Spherical Error" assumption is invoked for Gauss Markov. One of the parts of the assumption is that the variance is constant given $$X$$. The other is usually that the errors are uncorrelated with each other, given $$X$$. For example, on a regression of $$Y \sim X$$ on $$i \in \{1,\ldots,n\}$$:

$$E(\epsilon_i\epsilon_j \mid X)=0 \qquad i\neq j$$

I am failing to see where this assumption factors into the proof that the least squares estimator is BLUE. Is it necessary? If not, why is it assumed?

Glen_b has articulated correctly.

The assumptions

\begin{align} \mathbb{Var}[\varepsilon_i|\mathbf X] &= \sigma^2~\forall~i\in\{1, 2,\ldots, n\}, \\ \mathbb{Cov}[\varepsilon_i\varepsilon_j|\mathbf X] &= 0~\forall~i\ne j \end{align}

effectively can be summarised as

\begin{align} \mathbb{E}[\boldsymbol\varepsilon \boldsymbol\varepsilon^\mathsf T |\mathbf X] &= \begin{pmatrix} \mathbb{E}[\varepsilon_1\varepsilon_1|\mathbf X] & \mathbb{E}[\varepsilon_1\varepsilon_2|\mathbf X] & \ldots &\mathbb{E}[\varepsilon_1\varepsilon_n|\mathbf X]\\ \mathbb{E}[\varepsilon_2\varepsilon_1|\mathbf X] & \mathbb{E}[\varepsilon_2\varepsilon_2|\mathbf X] & \ldots &\mathbb{E}[\varepsilon_2\varepsilon_n|\mathbf X]\\ \vdots & \vdots & \ldots & \vdots\\ \mathbb{E}[\varepsilon_n\varepsilon_1|\mathbf X] & \mathbb{E}[\varepsilon_n\varepsilon_2|\mathbf X] & \ldots &\mathbb{E}[\varepsilon_n\varepsilon_n|\mathbf X] \end{pmatrix}\\[1em]&= \mathrm{Diag}(\sigma^2)\\[1em]&= \sigma^2\mathbb I_n. \end{align}

Gauss-Markov tells that any estimable function has a unique unbiased linear estimate which has minimum variance in the class of all unbiased linear estimates.

The proof, as devised by Scheffe, can be outlined as taking the unbiased estimator $$\mathbf a^\mathsf T\mathbf y$$ of, say $$\psi,$$ and decomposing $$\mathbf a = \mathbf a^\star + \mathbf b, ~\mathbf a^\star \in \mathbf V_r \subset \mathbf V_n, ~ \mathbf b \perp \mathbf V_r;$$ here $$\bf a^\star$$ is the projection of $$\bf a$$ on $$\mathbf V_r.$$ Then $$\psi = \mathbb E(\mathbf {a^\star}^\mathsf T\mathbf y) .$$

Coming to the variance part and where the assumption has actually been utilised,

\begin{align} \|\mathbf a\|^2 &= \|\mathbf a^\star\|^2 + \|\mathbf a-\mathbf a^\star\|^2, \\[1em] \mathbb{Var}(\mathbf a^\mathsf T\mathbf y) &= \sigma^2\|\mathbf a\|^2 \\[3pt]&= \underbrace{\sigma^2\|\mathbf a^\star\|^2}_{{\mathbb{Var}(\mathbf {a^\star}^\mathsf T\mathbf y) }} + \sigma^2\|\mathbf a-\mathbf a^\star\|^2; \end{align}

The rest follows. One can readily notice the necessity of the assumption.

## Reference:

The Analysis of Variance, Henry Scheffe, John Wiley & Sons, 1959.

Where exactly it's used depends on the proof you're looking at, but in this one it is used when going from the 2nd to the 3rd line of the $$\text{Var}(\tilde{\beta})$$ derivation.

• Are you referring to the 2nd to 3rd line where $Var(y) = \sigma^2$? If so, does that actually depend on $E(\epsilon_i\epsilon_j \mid X)=0 \qquad i\neq j$? Aug 15, 2022 at 4:24
• Note that $y$ is a vector, so $\text{Var}(y) = \sigma^{2}I$. So yes, it does. Aug 15, 2022 at 4:42
• Ah, so you are saying that if the condition didn't hold true we would not have the diagonal on the identity matrix, and instead there would be non-zero entries in the off-diagonals? Thank you. Aug 15, 2022 at 4:49