Inconsistent estimators in case of endogeneity

Question

Consider linear model $$y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \dots + \beta_kx_k +u$$ I've read in Wooldridge's "Econometric analysis for cross sectional and panel data" that if one of your regressors, say $x_k$, correlated with the error term then you end up with inconsistent estimators for all betas. I wonder why it is the case that all of them are inconsistent?

We know that $$\hat{\beta} - \beta = \left(\frac{1}{n}\sum_{i=1}^{n}x_i'x_i\right)^{-1}\left(\frac{1}{n}\sum_{i=1}^{n}x_i'u_i\right)$$ or in matrix notation $$\hat{\beta} - \beta = (X'X)^{-1}X'u$$

The true $\beta$ is given by $$\beta = \left(\mathbb{E}(x'_ix_i)\right)^{-1}(\mathbb{E}x'_iy)$$

In order for estimator to be consistent we need $$\hat{\beta} \xrightarrow{p} \beta$$ or $$\hat{\beta} - \beta \xrightarrow{p} 0$$ which is equivalent to $$\left(\frac{1}{n}\sum_{i=1}^{n}x_i'u_i\right) \xrightarrow{p} 0$$ or in matrix notation $$\text{plim}X'u = 0$$

If we zoom in then we have $$\begin{bmatrix} row \; x_{1} \\ row \; x_{2} \\ \vdots \\ row \; x_{k} \end{bmatrix} \begin{bmatrix} u_1\\ u_2\\ \vdots\\ u_n \end{bmatrix} = \begin{bmatrix} 0\\ 0\\ \vdots\\ 0 \end{bmatrix} $$

In case that $x_k$ correlated with $u$ we have $$\begin{bmatrix} row \; x_{1} \\ row \; x_{2} \\ \vdots \\ row \; x_{k} \end{bmatrix} \begin{bmatrix} u_1\\ u_2\\ \vdots\\ u_n \end{bmatrix} = \begin{bmatrix} 0\\ 0\\ \vdots\\ \varepsilon \end{bmatrix} $$

So, $\beta_k$ is inconsistent. But other estimates $\beta_0, \dots, \beta_{k-1} $ are consistent. At least that is the way I see it. Where is my mistake?

Answer 1

The mistake is, if $\text{plim}N^{-1}X'u = 0$, then we only need to assume $\text{plim}N^{-1}(X'X)^{-1}$ exists to prove consistency. But when $\text{plim}N^{-1}X'u \ne 0$, as is in the case when $x_k$ is endogenous (suppose $x_k$ is the last variable), we will need to check $\text{plim}N^{-1}(X'X)^{-1}$ to see if other $\beta$s are consistent.

If all other variables are orthogonal to $x_k$, then other $\beta$s will still be consistent. This is because the limit matrix of $(X'X)^{-1}$ will contain 0s for the last column and last row except the diagonal terms. Then right multiplied by $(0,\cdots,0,\epsilon)'$ gives 0 for other elements except $k$th.

Even in the case that only one variable $x_q$ is correlated with $x_k$, all other variables are not, but if some other variable $x_p$ is correlated with $x_q$, then we should not expect $\beta_p$ to be consistent. Intuitively but loosely, inconsistency spread through correlation.

Answer 2

Elaborating on @Paul 's answer, let $\text{plim}(X'X)^{-1} = W$ with typical element $[w^{ij}]$. Then, the expression for, say the $\hat \beta_0$ element of the vector $\hat \beta$ is, when only $X_k$ is correlated with $u$,

$$\text{plim} \hat \beta_0 = \beta_0 + w^{1k}\cdot E(X_ku)$$

So only if $w^{1k} = 0$, correlation won't spread.

Wooldridge page 62, ch 4 (1st ed.), provides an example where this is indeed the case and so the other eleents of the $\beta$-vector are consistently estimated. Assume that the cause of correlation is the existence of an unobservable variable $q$ in the error term

$$u = \gamma q + v$$

where $v$ is uncorrelated with the regressors.

Consider the Linear Projection of $q$ on the regressor matrix,

$$q = X\delta + r$$

and insert into the main regression to get

$$y = X(\beta + \gamma\delta) + \gamma r + v$$

By construction, $r$ and $u$ are uncorrelated with the regressors. So the probability limit of the OLS estimator will be

$$\text{plim}\hat \beta = \beta + \gamma\delta$$

We see that if any element of the $\delta$-vector is zero, OLS will consistently estimate the quantity of interest, the corresponding $\beta$ element.

But what does it mean if $\delta_j$ is zero? it means that in the presence of the other regressors, $X_j$ does not belong in the Linear Projection of $q$ on the regressor matrix.

Answer 3

Let's abstract from technicalities on Law of Large number and assume that for each $x_j$ for $j=1,...,k$ we have that $$\frac 1n\sum_{i=1}^{n} x_{i,j}u_i\to E(x_ju)$$ Now assme that for all but index $k$ we have that $E(x_ju)=0$. Let $E(x_ku)=\delta\neq0$. Let $X'X$ also have full rank, say you have an invertible limit for $\frac{1}{n}\sum_{i=1}^{n}x_i'x_i$ and call it $\Sigma_x$. Now $$\left(\frac{1}{n}\sum_{i=1}^{n}x_i'x_i\right)^{-1}\left(\frac{1}{n}\sum_{i=1}^{n}x_i'u_i\right)\to\Sigma_x^{-1}\times \begin{bmatrix} 0\\ 0\\ \vdots\\ \delta \end{bmatrix}$$ Open $\Sigma_x^{-1}$ as $$\Sigma_x^{-1}=\begin{bmatrix} [s_0 |r_0]\\ [s_1|r_1]\\ \vdots\\ [s_k|r_k] \end{bmatrix}$$ where $[s_0 |r_0]$ is the first row of $\Sigma_x^{-1}$ which we partition as a rwo vector of length $k$ and a scalar $r_0$. The other rows of $\Sigma_x^{-1}$ are also defined in the same fashion. As such we have $$\Sigma_x^{-1}\times \begin{bmatrix} 0\\ 0\\ \vdots\\ \delta \end{bmatrix}=\begin{bmatrix} r_0\times \delta\\ r_1\times \delta\\ \vdots\\ r_k\times \delta \end{bmatrix}=\delta\begin{bmatrix} r_0\\ r_1\\ \vdots\\ r_k \end{bmatrix}$$

Now if $\delta=0$ the latter limit will just bea vector of zeros implying asymptotic unbiasedness. But if $\delta \neq 0$ then the asymptotic bias for each element of $\beta$ is $\delta r_j$ for $j=0,1,...k$.