Consider linear model $$y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \dots + \beta_kx_k +u$$ I've read in Wooldridge's "Econometric analysis for cross sectional and panel data" that if one of your regressors, say $x_k$, correlated with the error term then you end up with inconsistent estimators for all betas. I wonder why it is the case that all of them are inconsistent?

We know that $$\hat{\beta} - \beta = \left(\frac{1}{n}\sum_{i=1}^{n}x_i'x_i\right)^{-1}\left(\frac{1}{n}\sum_{i=1}^{n}x_i'u_i\right)$$ or in matrix notation $$\hat{\beta} - \beta = (X'X)^{-1}X'u$$

The true $\beta$ is given by $$\beta = \left(\mathbb{E}(x'_ix_i)\right)^{-1}(\mathbb{E}x'_iy)$$

In order for estimator to be consistent we need $$\hat{\beta} \xrightarrow{p} \beta$$ or $$\hat{\beta} - \beta \xrightarrow{p} 0$$ which is equivalent to $$\left(\frac{1}{n}\sum_{i=1}^{n}x_i'u_i\right) \xrightarrow{p} 0$$ or in matrix notation $$\text{plim}X'u = 0$$

If we zoom in then we have $$\begin{bmatrix} row \; x_{1} \\ row \; x_{2} \\ \vdots \\ row \; x_{k} \end{bmatrix} \begin{bmatrix} u_1\\ u_2\\ \vdots\\ u_n \end{bmatrix} = \begin{bmatrix} 0\\ 0\\ \vdots\\ 0 \end{bmatrix} $$

In case that $x_k$ correlated with $u$ we have $$\begin{bmatrix} row \; x_{1} \\ row \; x_{2} \\ \vdots \\ row \; x_{k} \end{bmatrix} \begin{bmatrix} u_1\\ u_2\\ \vdots\\ u_n \end{bmatrix} = \begin{bmatrix} 0\\ 0\\ \vdots\\ \varepsilon \end{bmatrix} $$

So, $\beta_k$ is inconsistent. But other estimates $\beta_0, \dots, \beta_{k-1} $ are consistent. At least that is the way I see it. Where is my mistake?


The mistake is, if $\text{plim}N^{-1}X'u = 0$, then we only need to assume $\text{plim}N^{-1}(X'X)^{-1}$ exists to prove consistency. But when $\text{plim}N^{-1}X'u \ne 0$, as is in the case when $x_k$ is endogenous (suppose $x_k$ is the last variable), we will need to check $\text{plim}N^{-1}(X'X)^{-1}$ to see if other $\beta$s are consistent.

If all other variables are orthogonal to $x_k$, then other $\beta$s will still be consistent. This is because the limit matrix of $(X'X)^{-1}$ will contain 0s for the last column and last row except the diagonal terms. Then right multiplied by $(0,\cdots,0,\epsilon)'$ gives 0 for other elements except $k$th.

Even in the case that only one variable $x_q$ is correlated with $x_k$, all other variables are not, but if some other variable $x_p$ is correlated with $x_q$, then we should not expect $\beta_p$ to be consistent. Intuitively but loosely, inconsistency spread through correlation.

| cite | improve this answer | |

Let's abstract from technicalities on Law of Large number and assume that for each $x_j$ for $j=1,...,k$ we have that $$\frac 1n\sum_{i=1}^{n} x_{i,j}u_i\to E(x_ju)$$ Now assme that for all but index $k$ we have that $E(x_ju)=0$. Let $E(x_ku)=\delta\neq0$. Let $X'X$ also have full rank, say you have an invertible limit for $\frac{1}{n}\sum_{i=1}^{n}x_i'x_i$ and call it $\Sigma_x$. Now $$\left(\frac{1}{n}\sum_{i=1}^{n}x_i'x_i\right)^{-1}\left(\frac{1}{n}\sum_{i=1}^{n}x_i'u_i\right)\to\Sigma_x^{-1}\times \begin{bmatrix} 0\\ 0\\ \vdots\\ \delta \end{bmatrix}$$ Open $\Sigma_x^{-1}$ as $$\Sigma_x^{-1}=\begin{bmatrix} [s_0 |r_0]\\ [s_1|r_1]\\ \vdots\\ [s_k|r_k] \end{bmatrix}$$ where $[s_0 |r_0]$ is the first row of $\Sigma_x^{-1}$ which we partition as a rwo vector of length $k$ and a scalar $r_0$. The other rows of $\Sigma_x^{-1}$ are also defined in the same fashion. As such we have $$\Sigma_x^{-1}\times \begin{bmatrix} 0\\ 0\\ \vdots\\ \delta \end{bmatrix}=\begin{bmatrix} r_0\times \delta\\ r_1\times \delta\\ \vdots\\ r_k\times \delta \end{bmatrix}=\delta\begin{bmatrix} r_0\\ r_1\\ \vdots\\ r_k \end{bmatrix}$$

Now if $\delta=0$ the latter limit will just bea vector of zeros implying asymptotic unbiasedness. But if $\delta \neq 0$ then the asymptotic bias for each element of $\beta$ is $\delta r_j$ for $j=0,1,...k$.

| cite | improve this answer | |

Elaborating on @Paul 's answer, let $\text{plim}(X'X)^{-1} = W$ with typical element $[w^{ij}]$. Then, the expression for, say the $\hat \beta_0$ element of the vector $\hat \beta$ is, when only $X_k$ is correlated with $u$,

$$\text{plim} \hat \beta_0 = \beta_0 + w^{1k}\cdot E(X_ku)$$

So only if $w^{1k} = 0$, correlation won't spread.

Wooldridge page 62, ch 4 (1st ed.), provides an example where this is indeed the case and so the other eleents of the $\beta$-vector are consistently estimated. Assume that the cause of correlation is the existence of an unobservable variable $q$ in the error term

$$u = \gamma q + v$$

where $v$ is uncorrelated with the regressors.

Consider the Linear Projection of $q$ on the regressor matrix,

$$q = X\delta + r$$

and insert into the main regression to get

$$y = X(\beta + \gamma\delta) + \gamma r + v$$

By construction, $r$ and $u$ are uncorrelated with the regressors. So the probability limit of the OLS estimator will be

$$\text{plim}\hat \beta = \beta + \gamma\delta$$

We see that if any element of the $\delta$-vector is zero, OLS will consistently estimate the quantity of interest, the corresponding $\beta$ element.

But what does it mean if $\delta_j$ is zero? it means that in the presence of the other regressors, $X_j$ does not belong in the Linear Projection of $q$ on the regressor matrix.

| cite | improve this answer | |
  • 1
    $\begingroup$ Unfortunately, not so. The actual formula is $\hat{\beta} = \beta + (X^TX)^{-1}X^Te$, and the limit is $\beta$ if the limit of $X^Te = 0$. However, if only one element of $\lim X^Te \neq 0$, say element $i$, the inconsistency will propagate to all the $\hat{\beta_j}$ with nonzero correlation with $\hat{\beta_i}$, i.e., all $j$ such that $(X^TX)^{-1}_{ij} \neq 0$. $\endgroup$ – jbowman Oct 26 '16 at 22:48
  • $\begingroup$ @jbowman Figured it out, I will enhance and clarify my answer. The situation that Wooldridge describes in the specific passage is indeed such that we get consistent estimates of the other betas (because the inverse moment matrix is essentially assumed to contain the necessary zeros). But the book does not do a good job at explaining clearly the special nature of the case. $\endgroup$ – Alecos Papadopoulos Oct 27 '16 at 0:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.