Proof of consistency of OLS estimator under Heteroskedasticity

$$\DeclareMathOperator{\pl}{\operatorname{plim}}$$

Consider a general linear regression model with heteroskedastic errors $$\boldsymbol{y}=\boldsymbol{X}\boldsymbol{\beta}+\boldsymbol{u} \quad \text{with} \quad V(\boldsymbol{u}\vert \boldsymbol{X})=\sigma^2\boldsymbol{\Omega}$$ where $$\boldsymbol{\Omega}$$ is an arbitrary positive definite matrix. I want to proof that OLS is consistent under this setting and came up with two approaches:

One can express the OLS estimator as: $$\boldsymbol{\hat{\beta}}=\boldsymbol{\beta}+(\boldsymbol{X}^\top\boldsymbol{X})^{-1}\boldsymbol{X}^\top\boldsymbol{u} =\boldsymbol{\beta}+\left(\frac{\boldsymbol{X}^\top\boldsymbol{X}}{n}\right)^{-1}\frac{\boldsymbol{X}^\top\boldsymbol{u}}{n}$$ Assuming that $$\pl\left(\left(\frac{\boldsymbol{X}^\top\boldsymbol{X}}{n}\right)^{-1}\right)=\boldsymbol{\Sigma}_{\boldsymbol{x}\boldsymbol{x}}^{-1}$$ and noticing that $$\pl\left(\frac{\boldsymbol{X}^\top\boldsymbol{u}}{n}\right)=\boldsymbol{0}$$ yields: \begin{align*} \pl(\boldsymbol{\hat{\beta}})&=\boldsymbol{\beta}+\pl\left(\left(\frac{\boldsymbol{X}^\top\boldsymbol{X}}{n}\right)^{-1}\frac{\boldsymbol{X}^\top\boldsymbol{u}}{n}\right)\\ &=\boldsymbol{\beta}+\pl\left(\left(\frac{\boldsymbol{X}^\top\boldsymbol{X}}{n}\right)^{-1}\right)\pl\left(\frac{\boldsymbol{X}^\top\boldsymbol{u}}{n}\right)\\ &=\boldsymbol{\beta}+\boldsymbol{\Sigma}_{\boldsymbol{x}\boldsymbol{x}}^{-1}\boldsymbol{0}\\ &=\boldsymbol{\beta} \end{align*}

Another way to prove this is to note that: $$E(\boldsymbol{\hat{\beta}})=E(E(\boldsymbol{\hat{\beta}}\vert\boldsymbol{X}))=\boldsymbol{\beta}$$ The covariance matrix under heteroskedasticity is given by: \begin{align*} V(\boldsymbol{\hat{\beta}}\vert \boldsymbol{X})=\sigma^2(\boldsymbol{X}^\top\boldsymbol{X})^{-1}\boldsymbol{X}^\top\boldsymbol{\Omega}\boldsymbol{X}(\boldsymbol{X}^\top\boldsymbol{X})^{-1} =\frac{\sigma^2}{n}\left(\frac{\boldsymbol{X}^\top\boldsymbol{X}}{n}\right)^{-1}\left(\frac{\boldsymbol{X}^\top\boldsymbol{\Omega}\boldsymbol{X}}{n}\right)\left(\frac{\boldsymbol{X}^\top\boldsymbol{X}}{n}\right)^{-1} \end{align*} Thus, if $$\pl\left(\left(\frac{\boldsymbol{X}^\top\boldsymbol{X}}{n}\right)^{-1}\right)=\boldsymbol{\Sigma}_{\boldsymbol{x}\boldsymbol{x}}^{-1}$$ and $$\pl\left(\frac{\boldsymbol{X}^\top\boldsymbol{\Omega}\boldsymbol{X}}{n}\right)=\boldsymbol{\Sigma}_{\boldsymbol{x}\boldsymbol{\Omega}\boldsymbol{x}}$$, then $$V(\boldsymbol{\hat{\beta}}\vert \boldsymbol{X}) \rightarrow \boldsymbol{0}$$ as $$n \rightarrow \infty$$.

Now, my question. I wonder what is the relationship between the conditions $$\pl\left(\frac{\boldsymbol{X}^\top\boldsymbol{u}}{n}\right)=\boldsymbol{0}$$ and $$\pl\left(\frac{\boldsymbol{X}^\top\boldsymbol{\Omega}\boldsymbol{X}}{n}\right)=\boldsymbol{\Sigma}_{\boldsymbol{x}\boldsymbol{\Omega}\boldsymbol{x}}$$. In particular, in the second approach one implicitly uses that $$\pl\left(\frac{\boldsymbol{X}^\top\boldsymbol{u}}{n}\right)=\boldsymbol{0}$$, otherwise $$E(\boldsymbol{\hat{\beta}})\neq \boldsymbol{\beta}$$, however, I do not see why $$\pl\left(\frac{\boldsymbol{X}^\top\boldsymbol{u}}{n}\right)=\boldsymbol{0}$$ implies that $$\pl\left(\frac{\boldsymbol{X}^\top\boldsymbol{\Omega}\boldsymbol{X}}{n}\right)=\boldsymbol{\Sigma}_{\boldsymbol{x}\boldsymbol{\Omega}\boldsymbol{x}}$$. Both approaches seem reasonable to me - please correct me if I am wrong - and consequently both approaches should rely on the same assumptions.

1 Answer

$$\DeclareMathOperator{\pl}{\operatorname{plim}}$$

Firstly, a review of a simple result ($$\bf X$$ is nonstochastic):

Proposition $$1:$$ (cf. $$[\rm I], ~2.5, ~p. 65$$) If $$\lim_{n\to\infty}\frac{\mathbf {X^\top\Omega X}}{n}$$ is finite, then $$\hat{\boldsymbol\beta}$$ is consistent.

In \begin{align}\pl \hat{\boldsymbol\beta}&= \boldsymbol \beta + \lim \left(\frac{\mathbf{X^\top X}}{n}\right)^{-1}\pl \left(\frac{\mathbf X^\top\mathbf u}{n}\right)\tag 1, \label 1\end{align}

$$\left(\frac{\mathbf X^\top\mathbf u}{n}\right)$$ has mean zero mean and covariance matrix $$\sigma^2\left(\frac{\mathbf {X^\top\Omega X}}{n^2}\right)$$ which vanishes asymptotically as per the assumption of finiteness of $$\lim_{n\to\infty}\frac{\mathbf {X^\top\Omega X}}{n}$$ and this subsequently implies $$\pl \left(\frac{\mathbf X^\top\mathbf u}{n}\right) = \mathbf 0.$$

$$\blacksquare$$

Now, $$\operatorname{Var}[\hat{\boldsymbol\beta}|\mathbf X] = \frac{\sigma^2}{n}\left(\frac{\mathbf{X^\top X}}{n}\right)^{-1}\left(\frac{\mathbf {X^\top\Omega X}}{n}\right)\left(\frac{\mathbf{X^\top X}}{n}\right)^{-1}\tag 2\label 2.$$

In $$\eqref 2,$$ well-behaved regressors don't necessarily mean $$\frac{\sigma^2}{n}\left(\frac{\mathbf {X^\top\Omega X}}{n}\right)$$ will converge to zero (cf. $$[\rm II], ~9.3.2, p. 302$$).

However, (cf. $$\rm[II], 9.3.3.,~p. 304$$) it would be rather surprising $$\left(\frac{\mathbf{X^\top X}}{n}\right)$$ converges to a positive-definite matrix, but $$\left(\frac{\mathbf {X^\top\Omega X}}{n}\right)$$ doesn't.

Finally, consistency, in general, depends on $$\bf X$$ and $$\bf \Omega$$ both. As $$\rm [III]$$ shows,

Theorem $$1.$$ If $$\mathrm{ (a)}. ~\forall n, ~\lambda_\text{largest}(\Omega)$$ is bounded; $$\mathrm{(b)}. \lim_{n\to\infty}\lambda_\text{smallest}(\mathbf{X^\top X}) = \infty ,$$ then $$\hat{\boldsymbol\beta}$$ is consistent.

It follows from the inequality that for nonnegaive definite matrices $$\mathbf A,~\mathbf B ,~ \operatorname{tr}(\mathbf{AB})\leq \lambda_\text{largest}(\mathbf A)\operatorname{tr}(\mathbf{B})$$ in that

\begin{align}\operatorname{tr}\left[\left(\mathbf{X^\top X}\right)^{-1}\left(\mathbf {X^\top\Omega X}\right)\left(\mathbf{X^\top X}\right)^{-1}\right] &\leq \operatorname{tr}\left[\mathbf{ X\Omega}\left(\mathbf{X^\top X}\right)^{-2}\mathbf X^\top\right]\\ &\leq \lambda_\text{largest}(\mathbf \Omega)\operatorname{tr}(\mathbf{X^\top X})^{-1}\tag 3\label 3.\end{align}

The rest follows.

$$\blacksquare$$

References:

$$\rm [I]$$ Econometrics, Peter Schmidt, Taylor & Francis Group, $$1976.$$

$$\rm [II]$$ Econometric Analysis, William Greene, Pearson Education, $$2018.$$

$$\rm [III]$$ Advanced Econometrics, Takeshi Amemiya, Harvard University Press, $$1985,$$ sec. $$6.1.4,$$ pp. $$184-185.$$

• Thank you very much for this brilliant explanation. Jan 9, 2023 at 19:15