1
$\begingroup$

Let us assume the following data generating process:

(1) $ \ \ y = X\beta + u$

where $y$ and $u$ are $n\times 1$ vectors, $X$ is a $n\times k$-matrix with $rk(X)=k$ and $\beta$ is a $k\times 1$-vector of coefficients. For the OLS-estimator to be unbiased we need to assume that the conditional expectation of $u$ given $X$ i.e. $E(u|X)=0$ holds for (1). If the data are actually generated with (1) and $\beta_0$ is the true value for $\beta$ then we can write

$E(\hat{\beta}|X) = \beta_0 + (X^TX)^{-1}X^TE(u|X) = \beta_0$.

My question is: does the assumption $E(u|X)$ already imply by the law of large numbers (LLN) that $\text{plim}_{n\to\infty} \ \frac{1}{n}X^Tu =0$?

My guess is yes because by LLN $\text{plim}_{n\to\infty} \ \frac{1}{n}X^Tu = \text{lim}_{n\to\infty}\frac{1}{n}E(X^Tu) = 0$ ?

The reason for this question is that if the assumption $E(u|X)=0$ does imply that $\text{plim}_{n\to\infty} \ \frac{1}{n}X^Tu =0$ than the asymptotic consistency of $\hat{\beta}$ follows directly from this particular assumption.

$\endgroup$
  • 1
    $\begingroup$ Something is missing: "$E(u|X)$" is a number (or perhaps a vector), not a logical criterion, so it is nonsensical to write that it "holds." What did you intend? That $E(u|X)=0$? By the way, some explanation of your notation (at least to distinguish vectors from numbers) would make the question more readable, and therefore more likely to get good answers :-). $\endgroup$ – whuber Dec 7 '12 at 15:56
  • 1
    $\begingroup$ Thanks and sorry for my sloppy notation. I'll edit to get some clarity. $\endgroup$ – Druss2k Dec 7 '12 at 15:58
3
$\begingroup$

The canonical reference for this kind of thing is White (2001).

The model you have is $$ Y_i = \boldsymbol{X}_i'\boldsymbol{\beta}_0 + \varepsilon_i $$ together with the conditional exogeneity condition $\mathbb{E}(\varepsilon_i \mid \boldsymbol{X}_i) = 0$. Here $\boldsymbol{X}_i$ is a $k \times 1$ vector.

Your question is about the minimal conditions for the consistency of the OLS estimator. In particular, you would like to know if $\mathbb{E}(\varepsilon_i \mid \boldsymbol{X}_i) = 0$ is sufficient to prove that $\widehat{\boldsymbol{\beta}} \overset{p}{\to} \boldsymbol{\beta}_0$. We can go through the steps of proving the consistency to show that this condition is not sufficient.


  • We have that $$ \begin{align} \widehat{\boldsymbol{\beta}} &= \boldsymbol{\beta}_0 + \left(\frac{\mathbf{X}'\mathbf{X}}{n}\right)^{-1} \left(\frac{\mathbf{X}'\boldsymbol{\varepsilon}}{n}\right) \\ &= \boldsymbol{\beta}_0 +\left(\frac{\sum_{i=1}^n \boldsymbol{X}_i\boldsymbol{X}_i'}{n}\right)^{-1} \left(\frac{\sum_{i=1}^n\boldsymbol{X}_i\varepsilon_i}{n}\right) \end{align} $$ where $\mathbf{X} = [\boldsymbol{X}_1, \ldots, \boldsymbol{X}_n]'$, and $\boldsymbol{\varepsilon} = [\varepsilon_1, \ldots, \varepsilon_n]'$.
    As long as I can prove that $$ \frac{\sum_{i=1}^n\boldsymbol{X}_i\varepsilon_i}{n} \overset{p}{\to} \boldsymbol{0} $$ and that $$ \frac{\sum_{i=1}^n \boldsymbol{X}_i\boldsymbol{X}_i'}{n} \overset{p}{\to} \mathbf{m}_{\boldsymbol{X}\boldsymbol{X}} $$ where $\mathbf{m}_{\boldsymbol{X}\boldsymbol{X}}$ is a finite positive definite matrix, I can make two applications of Slutsky's theorem and get that $\widehat{\boldsymbol{\beta}} \overset{p}{\to} \boldsymbol{\beta}_0$. What guarantees this?

  • The first requirement is guaranteed to hold by an application of your favorite LLN (in the case of IID observations, you can use Khintchine's WLLN) to state that $$ \begin{align} \frac{\sum_{i=1}^n\boldsymbol{X}_i\varepsilon_i}{n} &\overset{p}{\to} \mathbb{E}(\boldsymbol{X}_i \varepsilon_i) \\ &= \boldsymbol{0} \end{align} $$ Note that your conditional exogeneity condition guarantees that $\mathbb{E}(\boldsymbol{X}_i \varepsilon_i)$ exists (why?) and that it is equal to $\boldsymbol{0}$. So far, we have not needed any additional assumptions.

  • The second requirement also requires use of an appropriate LLN to state that $$ \begin{align} \frac{\sum_{i=1}^n \boldsymbol{X}_i\boldsymbol{X}_i'}{n} &\overset{p}{\to} \mathbb{E}(\boldsymbol{X}_i\boldsymbol{X}_i') \\ &\equiv \mathbf{m}_{\boldsymbol{X}\boldsymbol{X}} \end{align} $$ where now we have to show that this convergence takes place and to a matrix with the required properties.
    An application of the LLN in this context requires the RHS of the limit to be finite (whence comes the finiteness of the $\mathbf{m}_{\boldsymbol{X}\boldsymbol{X}}$ matrix requirement).
    Next, even the pointwise full column rank of the design matrices $\mathbf{X}$, that is, the positive definiteness of $\mathbf{X}'\mathbf{X}$ does not guarantee that the limit is going to be positive definite, so that assumption will have to be made in addition.

Once you make these assumptions, then, you can claim consistency of the OLS estimator. Proving the consistency of the OLS estimator is possible under much weaker conditions of this type, for these, see White (2001, Ch. 2).

$\endgroup$
  • $\begingroup$ Thx (once again :-]) for your detailed answer. I forgot about the statement of the second condition, that $X^TX$ need to "converge" too. The answer to the (why?) is that using the law of iterated expectations we get that $E(ϵ_iX_i)=E(E(ϵ_iX_i|X_i))=E(X_iE(ϵ_i|X_i))=E(X_i⋅0)=0$. $\endgroup$ – Druss2k Dec 8 '12 at 19:35
  • $\begingroup$ This is just a tiny, pedantic thing to an otherwise amazing answer; but isn't the condition (from the beginning of your post) that $\mathbb{E}[\epsilon_i | \mathbf{X}]=0$? (i.e. conditional upon the design matrix at all $i$). From Greene textbook equation (2-6). $\endgroup$ – Jase Dec 13 '12 at 13:36
  • $\begingroup$ It is. But I guess fg nu just wrote it down for explanatory reasons. Or did u refer to my previous post? $\endgroup$ – Druss2k Dec 17 '12 at 12:17
  • $\begingroup$ @Jase, I do not think so. $E(\epsilon_i|X)=0$ is needed for unbiasedness, but not for consistency. Indeed, as the above derivation shows, not even $E(\epsilon_i|X_i)=0$ is necessary, as $E(\epsilon_i\cdot X_i)=0$ suffices. $\endgroup$ – Christoph Hanck Mar 7 '15 at 8:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.