0
$\begingroup$

Consider the linear regression model $y=X\beta+u$, where $y\in\mathbb R^T$ and $X\in\mathbb R^{T\times K}$. We make the following assumptions only:

  1. $\mathrm E[u]=0$;
  1. $\mathrm{Var}[u]=\sigma^2I_T$ for some $\sigma^2<\infty$.
  2. $X$ is either non-stochastic OR $X$ is independent of $u$ (my understanding is that this means that all columns of $X$ are independent of $u$ so that $X_{tk}$ is independent of $u_s$ for all $t,s=1,\dots,T$ and $k=1,\dots K$).

No further assumptions are made, i.e. the components of $u$ are not independent or identically distributed. Using Assumptions 1-3, we can show that the OLS estimator $\hat\beta=(X'X)^{-1}y=\beta+(X'X)^{-1}u$ is unbiased and has variance covariance $\sigma^2(X'X)^{-1}$. In addition to these two statistical properties, I want to show that $\hat\beta$ is consistent.

We add assumption

  1. $\mathrm{plim}(X'X/T)=Q$ if $X$ is stochastic and $\lim_{T\to\infty} (X'X)/T=Q$ if non-stochastic, for some positive-definite $Q$.

There are two ways to proceed. On the one hand, I can show $$\mathrm{plim\,}\hat\beta=\beta+Q^{-1}\mathrm{plim}\,(X'u/T).$$But I think here we cannot proceed because in order to use some form of the LLN to conclude that the second plim is $E(X'u)=0$ by assumption 3, we also need to make some additional assumption, like that $(X_{tk},u_t)$ is an i.i.d. sequence for all $k$. But this is not the case here.

Alternatively, I tried to use this result saying that an unbiased estimator whose variance tends to 0 is consistent. In this case the variance is $\sigma^2(X'X)^{-1}$ and with non-stochastic $X$ $$\lim_{T\to\infty}\sigma^2(X'X)^{-1}=\lim_{T\to\infty}\sigma^2(X'X)^{-1}T/T=\lim_{T\to\infty} \sigma^2Q^{-1}/T=0,$$ while with stochastic $X$ we have the same result with limits replaced by plims. This should "prove" that $\hat\beta$ is consistent according to the linked result. However, given the above I suspect that there is something wrong here, since $\hat\beta$ shouldn't be consistent with Assumptions 1-4 only. Could you help me find out what I'm doing wrong?

$\endgroup$
7
  • $\begingroup$ Why do you think $\hat{\beta}$ shouldn't be consistent with assumptions 1-4? $\endgroup$
    – jbowman
    Dec 20, 2022 at 20:50
  • $\begingroup$ I have been looking for proofs of consistency under 1-4 but couldn't find any because they all apply the LLN in some form and require i.i.d. errors. There was also some discussion about this here: stats.stackexchange.com/questions/65841/… $\endgroup$
    – Zugzwang14
    Dec 20, 2022 at 20:59
  • $\begingroup$ Yes, the stochastic case does require further assumptions. For the non-stochastic case, though, you're done. $\endgroup$
    – jbowman
    Dec 20, 2022 at 21:25
  • 2
    $\begingroup$ You don't need independent errors, but if you don't have them, you do need some other conditions to ensure the limit of the variance goes to zero. (Assume, for example, all the errors equal whatever the first error is, and you can see that the limit won't go to zero any more.) I am finally understanding what you'd like to see, but I don't believe it's possible w/o some additional assumptions. $\endgroup$
    – jbowman
    Dec 20, 2022 at 22:10
  • 1
    $\begingroup$ Your assumption 2 means the errors are uncorrelated (although not necessarily independent), which is what is needed to get to the variance-covariance matrix being $\sigma^2(X'X)^{-1}$. I think that's all that's needed. I'll look into this, meanwhile, maybe someone else will answer! $\endgroup$
    – jbowman
    Dec 20, 2022 at 22:39

1 Answer 1

2
+25
$\begingroup$

The fundamental sticking point you seem to be at is proving that $\mathrm{plim}(X'u/T) = 0$. We need to look at both the expected value and variance of $X'u/T$:

  1. $\mathbb{E} {X'u \over T} = 0$.

  2. $\mathbb{E} \left({X'u \over T}\right)\left({X'u \over T}\right)' = \mathbb{E} \left({X'uu' X \over T^2}\right) = {X'(\sigma^2\text{I})X \over T^2} = {\sigma^2 \over T}{X'X \over T}$

This follows straightforwardly from the assumptions.

Therefore,

  1. $\mathrm{lim}_{T \to \infty}\mathbb{E}\left({X'u \over T}\right)\left({X'u \over T}\right)' = \mathrm{lim}_{T \to \infty}{\sigma^2 \over T} {X'X \over T}= 0$

But statements 1 and 3 together imply that $\mathrm{plim}(X'u/T) = 0$.

It is essential that we have the limit of ${X'X \over T} = Q$ for this to work, otherwise, you could have a regressor that $\to \infty$ as $T \to \infty$ fast enough so that $X'u/T$ would be $O(T)$ or greater and the $\mathrm{plim}$ wouldn't go to zero.

This is essentially the same as the proof in Econometrics, by Peter Schmidt, which in my edition is ~ 250 pages of proofs.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.