2
$\begingroup$

Consider a model

$$\mathbf y=\beta_0+\mathbf x_1\beta_1+\mathbf x_2\beta_2+\mathbf e$$

and assume $\mathbb E[\mathbf e\mid\mathbf X]=0$. Under this scenario, the OLS estimator is a consistent estimator. I have no trouble showing this explicitly, and otherwise when $\mathbb E[\mathbf e\mid\mathbf X]\neq0$.

However, I can't understand how I'm supposed to think when I'm given questions like:

  1. Suppose $\mathbf x_1$ is the age variable, and suppose we exclude data from the sample for which $\mathbf x_1<25$. Would the OLS estimator still be consistent? (targeting a covariate)
  2. Suppose $\mathbf y$ is a person's height in cm, and suppose we exclude data for which $\mathbf y<85$. Would the OLS estimator still be consistent? (targeting a dependent variable)

For both cases I would go with a 'no'. My thinking is that after the 'filtering' the data of our sample is no longer representative of the data for the whole population. But I may as well be wrong.

$\endgroup$
4
  • 2
    $\begingroup$ Although the truncated data are not representative of the population, that's not really the question. The question is whether the coefficients of the regression in the truncated population ought to be the same as the coefficients in the whole population. To this end, imagine drawing a scatterplot of an entire populatoin. Ask yourself how removing certain points from the scatterplot could affect the linear regression. $\endgroup$
    – whuber
    Commented Feb 10, 2021 at 16:09
  • 1
    $\begingroup$ If we removed certain points from the population, drawn from it a sample, and the fitted a model, the estimates will be consistent but only for that specific 'truncated' population, and the general population. Am I getting this right? $\endgroup$
    – E.Z
    Commented Feb 10, 2021 at 16:12
  • 2
    $\begingroup$ I can't tell what you are claiming. Please note that my point doesn't concern samples at all: it suggests you consider the true values of the regression coefficients within the population or truncated versions of it. The idea is that if the true values in the truncated population are the same as the true values in the population, then because OLS for samples from the truncated population are consistent, a fortiori they will be consistent for the population. $\endgroup$
    – whuber
    Commented Feb 10, 2021 at 16:22
  • 2
    $\begingroup$ Okay, now I'm getting what you're saying, thanks. $\endgroup$
    – E.Z
    Commented Feb 10, 2021 at 16:23

1 Answer 1

2
$\begingroup$

Your answer to question 2 is correct, provided here are actually individuals with $y \geq 85$. The easiest way to see this is that this truncation would at the very least decrease the mean value of $y$, so even if you cooked up the data in such a way that $\beta_1, \beta_2$ from such a regression stayed exactly the same, then $\beta_0$ would still have to decrease to reflect the fact that the average value of the population has shifted down.

Your answer to question 1, however is incorrect if the assumptions you have stated are taken to be literally true. In particular, the assumption that $\mathbb E[e | X] = 0$ is equivalent to the assumption that $$\mathbb E[y | X] = \beta_0 + x_1\beta_1 + x_2\beta_2$$ This assumption is quite strong! It's saying that the conditional expectation of $y$ given $X$ is actually linear in $X$ (i.e. not just approximated by a linear function). Put another way, this assumption is saying that extrapolation is valid; how $y$ behaves on average for $X$ in some neighborhood is completely informative about how $y$ behaves for $X$ in some completely different neighborhood. As a result, fitting the model only with $x_1 < 25$ will typically give consistent estimates of the parameters, provided there is enough variation to identify everything, and provided the (very strong!) linear functional form is literally true.

Contrast this, for example, to the weaker assumption that merely $\mathbb E[eX] = 0$ (i.e. the error is uncorrelated with $X$). In this case, we can show that the "true" $\beta$ that is being estimated by OLS is the one that solves the population least squares problem: $$\beta_0, \beta_1, \beta_2 = \underset{b_0, b_1, b_2}{\mathrm{argmin}}\ \mathbb E\left[(y - b_0 - x_1 b_1 - x_2 b_2)^2\right]$$

Under this weaker assumption, we have that $\beta$ parameterizes the "best" linear approximation to the conditional expectation function (where "best" means minimizing a mean squared error objective) of $y$. Notably, the definition of "best" is now dependent on the distribution of $X$ since that expectation is with respect to the entire joint distribution of $Y,X$. Put another way, this weaker assumption drops the strong condition implied by $\mathbb E[e | X] = 0$ that extrapolation is valid. In that case, you are absolutely right in stating that subsetting to $x_1 < 25$ would potentially give different results compared to not doing this subsetting.

$\endgroup$
1
  • 2
    $\begingroup$ I think I agree with the conclusion of your first paragraph but not with the argumentation/explanation. Thresholding some $x$ would also change the mean of $y$ but need not change the regression coefficients (say, $\beta_0$). Thresholding $y$ is a different matter for a different reason. You would need to explain how and why. $\endgroup$ Commented Feb 10, 2021 at 19:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.