5
$\begingroup$

Suppose I regress (with OLS) $y$ on $x_1$ and $x_2$. Suppose I have i.i.d. sample of size n, and that $x_1$ is observed with error but $y$ and $x_2$ are observed without error. What is the probability limit of the estimated coefficient on $x_1$?

Let us suppose for tractability that the measurement error of $x_1$ is "classical". That is the measurement error is normally distributed with mean 0 and is uncorrelated with $x_2$ or the error term.

$\endgroup$
9
  • $\begingroup$ What do you mean by "probability limit"? Are the variances of the equation error and/or the measurement error in $x_1$ known? Or maybe the ratio of their variances? Also, OLS is simply not the correct regression model if you have errors in variables. I believe that what you're looking for is called errors-in-variables regression. The exact method to use depends on your answers to my questions above. $\endgroup$
    – Eike P.
    Commented Jul 24, 2020 at 18:22
  • $\begingroup$ @jhin thank you for looking! I'm not trying to correct for the measurement error, so it's not the case that the researcher knows the variance; or you can pretend that they know them but do not use it in their estimation. Rather than correct measurement error, I want to understand the theoretical consequences. Regarding probability limit, we can view the OLS estimator as a random variable that converges in probability (en.wikipedia.org/wiki/…) as n increases. The thing it converges to is what I'm calling the "probability limit". $\endgroup$
    – Xu Wang
    Commented Jul 24, 2020 at 19:27
  • 1
    $\begingroup$ What you describe is a multiple regression (more than one independent variable), not a multivariate regression (more than one dependent variable). I have edited the title to correct. $\endgroup$
    – Alexis
    Commented Jul 30, 2020 at 1:46
  • 1
    $\begingroup$ @Alexis thank you for your edit, and more importantly for kindly explaining the edit. I am embarrassed I did not know that, and appreciate having learnt. $\endgroup$
    – Xu Wang
    Commented Jul 30, 2020 at 16:02
  • 1
    $\begingroup$ @XuWang No need for embarasment! We are all learning. Welcome to CV. :) $\endgroup$
    – Alexis
    Commented Jul 30, 2020 at 16:14

3 Answers 3

5
+50
$\begingroup$

Suppose you true matrix is $X^*=\begin{bmatrix} x_1^{*} & x_2 \end{bmatrix}$, but you observe $x_1=x_1^*+v$.

Then the OLS coefficient on $x_1$ has the following probability limit:

$$\mathbf{plim} \hat \beta_{x_1|x_2}=\beta \left[1-\frac{\sigma^2_v}{\sigma^2_{x_1^*}\cdot(1-R^2_{x_1^*,x_2})+\sigma^2_v} \right]=\beta \left[1-\frac{1}{1+\frac{\sigma^2_{x_1^*}}{\sigma^2_v}\cdot(1-R^2_{x_1^*,x_2})} \right]$$

where $R^2_{x_1^*,x_2}$ denotes the $R^2$ from the auxiliary regression of $x_1^*$ on $x_2$.

This means the coefficient is still attenuated, but generally less so than in the single regressor case. The bias gets worse as collinearity with $x_2$ increases.

Here $x_2$ can contain more than one variable measured without error, so this formula is pretty general. The coefficient(s) measured without error will be inconsistent in the direction determined by $\Sigma_{X^*X^*}$.

You can find this formula (without proof, but surrounded by much auxiliary wisdom) in equation (5) in Bound, John & Brown, Charles & Mathiowetz, Nancy, 2001. "Measurement error in survey data," Handbook of Econometrics, edition 1, volume 5, chapter 59, pages 3705-3843.

They cite these two older papers:

  • Levi, M.D. (1973), "Errors in the variables bias in the presence of correctly measured variables", Econometrica 41:985−986.
  • Garber, S., and S. Klepper (1980), “Extending the classical normal errors-in-variables model”, Econometrica 48:1541−1546.
$\endgroup$
2
$\begingroup$

The solution to this problem is in Wooldridge's "Introductory Econometrics" (Chapter 9 Section "Measurement Error in an Explanatory Variable", p320 in the 2012 version) and in Wooldridge's "Econometric analysis of cross-section and_panel data" (Section 4.4.2, p73 in the 2002 version). Here is the takeaway.

Consider the multiple regression model with a single explanatory variable $x^*_K$ measured with error :

$$y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + ... + \beta_K x^*_K + \nu$$

And with "classical" assumptions, mainly that $\nu$ is uncorrelated to $x^*_K$ and $\nu$ is uncorrelated to $x_K$.

The measurement error is $e_K = x_K - x^*_K$ with $\text{E}(e_k) = 0$. The classical assumption implies that $\nu$ is uncorrelated to $e_K$

We want to replace $x^*_K$ with $x_K$ and see how this affects OLS estimators, w.r.t. assumptions on the relationship between the measurement error $e_k$ and $x^*_K$ and $x_K$.

The first case, which is not the OP case but I present briefly for the sake of completeness, is when $\text{Cov}(e_K, x_K) = 0$. Here OLS using $x_K$ instead of $x^*_K$ provides consistent estimators even if it inflates the error variance of the estimations (and thus of the estimators).

The case of interest is when $\text{Cov}(e_K, x^*_K) = 0$ and is called "classical in variables error" in the econometric literature. Here :

$$\text{Cov}(e_K, x_K) = \text{E}(e_Kx_K) = \text{E}(e_Kx^*_K)+ \text{E}(e^2_K) = \sigma^{2}_{e_{K}} $$

and :

$$ \text{plim}(\hat{\beta}_k) = \beta_K \left( \frac{\sigma^{2}_{r^{*}_{K}}}{\sigma^{2}_{r^{*}_{K}}+ \sigma^{2}_{e_{K}}} \right) = \beta_KA_K $$

where $r_K$ is error in :

$$ x^*_K = \delta_0 + \delta_1 x_1 + \delta_2 x_2 + ... \delta_{K-1} x_{K-1} + r^*_K $$

$A_K$ is always between 0 and 1 and is called the attenuation bias: If $\beta_K$ is positive (reps. negative), $\hat{\beta}_K$ will tend to underestimate (reps. underestimate) $\beta_K$.

In the multivariate regression, it is the variance of $x^*_K$ after controlling (netting) for the effects of the other explanatory variables, that affect the attenuation bias. This latter is worse as $x^*_K$ is colinear with the other variables.

In the case where $K=1$, i.e., the simple regression model where there is only one explanatory variable which is measured with error. In this case :

$$\text{plim}(\hat{\beta}_1) = \beta_1 \left( \frac{ \sigma^{2}_{x^*_1} }{\sigma^{2}_{x^*_1} + \sigma^{2}_{e_1}} \right)$$

The attenuation term, always between 0 and 1 becomes closer to 1 as $\sigma^{2}_{e_1}$ shrinks relatively to $\sigma^{2}_{x^*_1}$. Note that in this special case, $r^*_K = x^*_1$.

The $\text{plim}(\beta_j)$ for $j \neq K$ is complicated to derive in this framework, except in the case where $x^*_k$ is uncorrelated to other $x_j$, thus $x_k$ is uncorrelated to other $x_j$, which leads to $\text{plim}(\hat{\beta}_j)=\beta_j$.

$\endgroup$
1
  • 1
    $\begingroup$ It has been a pleasure to put this stuff down. Hope it will help you :) $\endgroup$
    – SoufianeK
    Commented Jul 31, 2020 at 16:16
1
$\begingroup$

In the situation that you describe, the true model is like this:

$y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + u$

now, you can observe $y$ and $x_2$ but you cannot observe $x_1$. However you can observe $z = x_1 + \epsilon$

moreover we assume that $\rho(\epsilon,u)=0$

So, if we consider the simplification where $\beta_2 =0$ is possible to show that the OLS estimator for $\beta_1$ is like

$\theta_1 = \beta_1 V[x_1]/(V[x_1] + V[\epsilon]) $

Then the absolute value of $\theta_1$, in expected term and/or plim, is lower than of $\beta_1$. Then $\theta_1$ is biased (incorrect and inconsistent) for $\beta_1$. This kind of bias is known as attenuation bias. More $V[\epsilon]$ increase, more serious the problem become.

Now, for multivariate case the matrix algebra notation is usually used. Then in vector form we achieve $E[\theta]\neq \beta$ and/or plim $\theta \neq\beta$
Note that here you can consider that even if only one variable is endogenous, for measurement error or other problems, all parameters become biased. The direction of bias for any $\theta_i$ depends from correlations among variables and the sign of the first moments. Special cases exist, for example if the variables are all orthogonal the bias does not spread.

In your case, two variables ($\beta_1$ and $\beta_2$ different from $0$), you can estimate a regression like

$y = \theta_0 + \theta_1 z + \theta_2 x_2 + v$

here $\theta_1$ suffer from attenuation bias (on $\beta_1$) but also $\theta_2$ is biased (for $\beta_2$). In special case where $z$ and $x_2$ are orthogonal, for $\theta_1$ the problem remain but $\theta_2$ become correct and consistent.

$\endgroup$
10
  • $\begingroup$ Thank you for your answer. How can you be sure that $\theta_1$ suffers from attenuation bias? Can you provide the exact probability limit of the estimator? $\endgroup$
    – Xu Wang
    Commented Jul 27, 2020 at 18:58
  • $\begingroup$ The formula above for $\theta_1$ is its probability limit. In short form plim $\theta_1 = \beta_1 A$ where $0<A<1$. Then $A$ is an attenuator. $\endgroup$
    – markowitz
    Commented Jul 28, 2020 at 8:54
  • $\begingroup$ But you only derived $A$ for the case where we assume $\beta_2$ is 0. The question is about when we cannot make the assumption that $\beta_2$ is 0. I'm sorry if that was not clear. $\endgroup$
    – Xu Wang
    Commented Jul 28, 2020 at 17:09
  • $\begingroup$ The formula is given for single regressor case, right. However I gave some insight about what can happen in multivariate case. The math detail about the two regressors case, is only as matter of algebra. I’m sorry but I do not have time now to go deeper. Moreover note that if you don’t love algebra you can easily simulate and check for any combination of correlation and moments sign. $\endgroup$
    – markowitz
    Commented Jul 29, 2020 at 8:26
  • 1
    $\begingroup$ In my opinion the Dimitriy explanation is the more convincing one. $\endgroup$
    – markowitz
    Commented Jul 30, 2020 at 16:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.