0
$\begingroup$

$\DeclareMathOperator{\pl}{\operatorname{plim}}$

Consider a general linear regression model with heteroskedastic errors $$ \boldsymbol{y}=\boldsymbol{X}\boldsymbol{\beta}+\boldsymbol{u} \quad \text{with} \quad V(\boldsymbol{u}\vert \boldsymbol{X})=\sigma^2\boldsymbol{\Omega} $$ where $\boldsymbol{\Omega}$ is an arbitrary positive definite matrix. I want to proof that OLS is consistent under this setting and came up with two approaches:


One can express the OLS estimator as: $$ \boldsymbol{\hat{\beta}}=\boldsymbol{\beta}+(\boldsymbol{X}^\top\boldsymbol{X})^{-1}\boldsymbol{X}^\top\boldsymbol{u} =\boldsymbol{\beta}+\left(\frac{\boldsymbol{X}^\top\boldsymbol{X}}{n}\right)^{-1}\frac{\boldsymbol{X}^\top\boldsymbol{u}}{n} $$ Assuming that $\pl\left(\left(\frac{\boldsymbol{X}^\top\boldsymbol{X}}{n}\right)^{-1}\right)=\boldsymbol{\Sigma}_{\boldsymbol{x}\boldsymbol{x}}^{-1}$ and noticing that $\pl\left(\frac{\boldsymbol{X}^\top\boldsymbol{u}}{n}\right)=\boldsymbol{0}$ yields: \begin{align*} \pl(\boldsymbol{\hat{\beta}})&=\boldsymbol{\beta}+\pl\left(\left(\frac{\boldsymbol{X}^\top\boldsymbol{X}}{n}\right)^{-1}\frac{\boldsymbol{X}^\top\boldsymbol{u}}{n}\right)\\ &=\boldsymbol{\beta}+\pl\left(\left(\frac{\boldsymbol{X}^\top\boldsymbol{X}}{n}\right)^{-1}\right)\pl\left(\frac{\boldsymbol{X}^\top\boldsymbol{u}}{n}\right)\\ &=\boldsymbol{\beta}+\boldsymbol{\Sigma}_{\boldsymbol{x}\boldsymbol{x}}^{-1}\boldsymbol{0}\\ &=\boldsymbol{\beta} \end{align*}


Another way to prove this is to note that: $$ E(\boldsymbol{\hat{\beta}})=E(E(\boldsymbol{\hat{\beta}}\vert\boldsymbol{X}))=\boldsymbol{\beta} $$ The covariance matrix under heteroskedasticity is given by: \begin{align*} V(\boldsymbol{\hat{\beta}}\vert \boldsymbol{X})=\sigma^2(\boldsymbol{X}^\top\boldsymbol{X})^{-1}\boldsymbol{X}^\top\boldsymbol{\Omega}\boldsymbol{X}(\boldsymbol{X}^\top\boldsymbol{X})^{-1} =\frac{\sigma^2}{n}\left(\frac{\boldsymbol{X}^\top\boldsymbol{X}}{n}\right)^{-1}\left(\frac{\boldsymbol{X}^\top\boldsymbol{\Omega}\boldsymbol{X}}{n}\right)\left(\frac{\boldsymbol{X}^\top\boldsymbol{X}}{n}\right)^{-1} \end{align*} Thus, if $\pl\left(\left(\frac{\boldsymbol{X}^\top\boldsymbol{X}}{n}\right)^{-1}\right)=\boldsymbol{\Sigma}_{\boldsymbol{x}\boldsymbol{x}}^{-1}$ and $\pl\left(\frac{\boldsymbol{X}^\top\boldsymbol{\Omega}\boldsymbol{X}}{n}\right)=\boldsymbol{\Sigma}_{\boldsymbol{x}\boldsymbol{\Omega}\boldsymbol{x}}$, then $$ V(\boldsymbol{\hat{\beta}}\vert \boldsymbol{X}) \rightarrow \boldsymbol{0} $$ as $n \rightarrow \infty$.


Now, my question. I wonder what is the relationship between the conditions $\pl\left(\frac{\boldsymbol{X}^\top\boldsymbol{u}}{n}\right)=\boldsymbol{0}$ and $\pl\left(\frac{\boldsymbol{X}^\top\boldsymbol{\Omega}\boldsymbol{X}}{n}\right)=\boldsymbol{\Sigma}_{\boldsymbol{x}\boldsymbol{\Omega}\boldsymbol{x}}$. In particular, in the second approach one implicitly uses that $\pl\left(\frac{\boldsymbol{X}^\top\boldsymbol{u}}{n}\right)=\boldsymbol{0}$, otherwise $E(\boldsymbol{\hat{\beta}})\neq \boldsymbol{\beta}$, however, I do not see why $\pl\left(\frac{\boldsymbol{X}^\top\boldsymbol{u}}{n}\right)=\boldsymbol{0}$ implies that $\pl\left(\frac{\boldsymbol{X}^\top\boldsymbol{\Omega}\boldsymbol{X}}{n}\right)=\boldsymbol{\Sigma}_{\boldsymbol{x}\boldsymbol{\Omega}\boldsymbol{x}}$. Both approaches seem reasonable to me - please correct me if I am wrong - and consequently both approaches should rely on the same assumptions.

$\endgroup$

1 Answer 1

1
$\begingroup$

$\DeclareMathOperator{\pl}{\operatorname{plim}}$

Firstly, a review of a simple result ($\bf X$ is nonstochastic):

Proposition $1:$ (cf. $[\rm I], ~2.5, ~p. 65$) If $\lim_{n\to\infty}\frac{\mathbf {X^\top\Omega X}}{n}$ is finite, then $\hat{\boldsymbol\beta}$ is consistent.

In \begin{align}\pl \hat{\boldsymbol\beta}&= \boldsymbol \beta + \lim \left(\frac{\mathbf{X^\top X}}{n}\right)^{-1}\pl \left(\frac{\mathbf X^\top\mathbf u}{n}\right)\tag 1, \label 1\end{align}

$\left(\frac{\mathbf X^\top\mathbf u}{n}\right)$ has mean zero mean and covariance matrix $\sigma^2\left(\frac{\mathbf {X^\top\Omega X}}{n^2}\right)$ which vanishes asymptotically as per the assumption of finiteness of $\lim_{n\to\infty}\frac{\mathbf {X^\top\Omega X}}{n}$ and this subsequently implies $\pl \left(\frac{\mathbf X^\top\mathbf u}{n}\right) = \mathbf 0.$

$\blacksquare$

Now, $$\operatorname{Var}[\hat{\boldsymbol\beta}|\mathbf X] = \frac{\sigma^2}{n}\left(\frac{\mathbf{X^\top X}}{n}\right)^{-1}\left(\frac{\mathbf {X^\top\Omega X}}{n}\right)\left(\frac{\mathbf{X^\top X}}{n}\right)^{-1}\tag 2\label 2.$$

In $\eqref 2,$ well-behaved regressors don't necessarily mean $ \frac{\sigma^2}{n}\left(\frac{\mathbf {X^\top\Omega X}}{n}\right)$ will converge to zero (cf. $[\rm II], ~9.3.2, p. 302 $).

However, (cf. $\rm[II], 9.3.3.,~p. 304$) it would be rather surprising $\left(\frac{\mathbf{X^\top X}}{n}\right)$ converges to a positive-definite matrix, but $\left(\frac{\mathbf {X^\top\Omega X}}{n}\right)$ doesn't.

Finally, consistency, in general, depends on $\bf X$ and $\bf \Omega$ both. As $\rm [III]$ shows,

Theorem $1.$ If $\mathrm{ (a)}. ~\forall n, ~\lambda_\text{largest}(\Omega)$ is bounded; $\mathrm{(b)}. \lim_{n\to\infty}\lambda_\text{smallest}(\mathbf{X^\top X}) = \infty ,$ then $\hat{\boldsymbol\beta}$ is consistent.

It follows from the inequality that for nonnegaive definite matrices $\mathbf A,~\mathbf B ,~ \operatorname{tr}(\mathbf{AB})\leq \lambda_\text{largest}(\mathbf A)\operatorname{tr}(\mathbf{B})$ in that

\begin{align}\operatorname{tr}\left[\left(\mathbf{X^\top X}\right)^{-1}\left(\mathbf {X^\top\Omega X}\right)\left(\mathbf{X^\top X}\right)^{-1}\right] &\leq \operatorname{tr}\left[\mathbf{ X\Omega}\left(\mathbf{X^\top X}\right)^{-2}\mathbf X^\top\right]\\ &\leq \lambda_\text{largest}(\mathbf \Omega)\operatorname{tr}(\mathbf{X^\top X})^{-1}\tag 3\label 3.\end{align}

The rest follows.

$\blacksquare$


References:

$\rm [I]$ Econometrics, Peter Schmidt, Taylor & Francis Group, $1976.$

$\rm [II]$ Econometric Analysis, William Greene, Pearson Education, $2018.$

$\rm [III]$ Advanced Econometrics, Takeshi Amemiya, Harvard University Press, $1985,$ sec. $6.1.4,$ pp. $184-185.$

$\endgroup$
1
  • 1
    $\begingroup$ Thank you very much for this brilliant explanation. $\endgroup$
    – Count
    Jan 9, 2023 at 19:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.