3
$\begingroup$

I often see that the "Spherical Error" assumption is invoked for Gauss Markov. One of the parts of the assumption is that the variance is constant given $X$. The other is usually that the errors are uncorrelated with each other, given $X$. For example, on a regression of $Y \sim X$ on $i \in \{1,\ldots,n\}$:

$$ E(\epsilon_i\epsilon_j \mid X)=0 \qquad i\neq j $$

I am failing to see where this assumption factors into the proof that the least squares estimator is BLUE. Is it necessary? If not, why is it assumed?

$\endgroup$

2 Answers 2

9
$\begingroup$

Glen_b has articulated correctly.

The assumptions

\begin{align} \mathbb{Var}[\varepsilon_i|\mathbf X] &= \sigma^2~\forall~i\in\{1, 2,\ldots, n\}, \\ \mathbb{Cov}[\varepsilon_i\varepsilon_j|\mathbf X] &= 0~\forall~i\ne j \end{align}

effectively can be summarised as

\begin{align} \mathbb{E}[\boldsymbol\varepsilon \boldsymbol\varepsilon^\mathsf T |\mathbf X] &= \begin{pmatrix} \mathbb{E}[\varepsilon_1\varepsilon_1|\mathbf X] & \mathbb{E}[\varepsilon_1\varepsilon_2|\mathbf X] & \ldots &\mathbb{E}[\varepsilon_1\varepsilon_n|\mathbf X]\\ \mathbb{E}[\varepsilon_2\varepsilon_1|\mathbf X] & \mathbb{E}[\varepsilon_2\varepsilon_2|\mathbf X] & \ldots &\mathbb{E}[\varepsilon_2\varepsilon_n|\mathbf X]\\ \vdots & \vdots & \ldots & \vdots\\ \mathbb{E}[\varepsilon_n\varepsilon_1|\mathbf X] & \mathbb{E}[\varepsilon_n\varepsilon_2|\mathbf X] & \ldots &\mathbb{E}[\varepsilon_n\varepsilon_n|\mathbf X] \end{pmatrix}\\[1em]&= \mathrm{Diag}(\sigma^2)\\[1em]&= \sigma^2\mathbb I_n. \end{align}

Gauss-Markov tells that any estimable function has a unique unbiased linear estimate which has minimum variance in the class of all unbiased linear estimates.

The proof, as devised by Scheffe, can be outlined as taking the unbiased estimator $\mathbf a^\mathsf T\mathbf y$ of, say $\psi, $ and decomposing $\mathbf a = \mathbf a^\star + \mathbf b, ~\mathbf a^\star \in \mathbf V_r \subset \mathbf V_n, ~ \mathbf b \perp \mathbf V_r; $ here $\bf a^\star$ is the projection of $\bf a$ on $\mathbf V_r.$ Then $$\psi = \mathbb E(\mathbf {a^\star}^\mathsf T\mathbf y) .$$

Coming to the variance part and where the assumption has actually been utilised,

\begin{align} \|\mathbf a\|^2 &= \|\mathbf a^\star\|^2 + \|\mathbf a-\mathbf a^\star\|^2, \\[1em] \mathbb{Var}(\mathbf a^\mathsf T\mathbf y) &= \sigma^2\|\mathbf a\|^2 \\[3pt]&= \underbrace{\sigma^2\|\mathbf a^\star\|^2}_{{\mathbb{Var}(\mathbf {a^\star}^\mathsf T\mathbf y) }} + \sigma^2\|\mathbf a-\mathbf a^\star\|^2; \end{align}

The rest follows. One can readily notice the necessity of the assumption.


Reference:

The Analysis of Variance, Henry Scheffe, John Wiley & Sons, 1959.

$\endgroup$
1
$\begingroup$

Where exactly it's used depends on the proof you're looking at, but in this one it is used when going from the 2nd to the 3rd line of the $\text{Var}(\tilde{\beta})$ derivation.

$\endgroup$
3
  • $\begingroup$ Are you referring to the 2nd to 3rd line where $Var(y) = \sigma^2$? If so, does that actually depend on $E(\epsilon_i\epsilon_j \mid X)=0 \qquad i\neq j$? $\endgroup$
    – user321627
    Commented Aug 15, 2022 at 4:24
  • $\begingroup$ Note that $y$ is a vector, so $\text{Var}(y) = \sigma^{2}I$. So yes, it does. $\endgroup$
    – Glen_b
    Commented Aug 15, 2022 at 4:42
  • $\begingroup$ Ah, so you are saying that if the condition didn't hold true we would not have the diagonal on the identity matrix, and instead there would be non-zero entries in the off-diagonals? Thank you. $\endgroup$
    – user321627
    Commented Aug 15, 2022 at 4:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.