3
$\begingroup$

I've read Zero conditional mean assumption (how can in not hold?).

In linear regression, we assume the model follows $$y_i = \beta_0 + \beta_1 x_i + \epsilon_i $$ under the assumption that $\mathbb{E}(\epsilon|X) = 0$.

However, I don't see why we must have that assumption, even if $\epsilon_i$ and $x_i$ are correlated.

Because if $\mathbb{E}(\epsilon|X) = c \neq 0$, then why can't we just assume another model $$y_i = \underline{\beta}_0 + \beta_1 x_i + \underline{\epsilon}_i$$ where $\underline{\beta}_0 = \beta_0 + c$ and $\underline{\epsilon}_i = \epsilon_i - c$

and applying the typical algorithm, we can find estimates for these new parameters?

$\endgroup$
6
  • $\begingroup$ How could your data distinguish between those two models? $\endgroup$
    – whuber
    Commented Sep 24, 2019 at 11:25
  • $\begingroup$ I'm not sure, is it important to distinguish them? $\endgroup$ Commented Sep 24, 2019 at 11:30
  • 2
    $\begingroup$ When you posit models that cannot possibly be distinguished by data, you have an "identifiability" problem. Although there's no mathematical issue with that, it can create problems with software (if it is searching for "the" optimum solution, what happens when there's an entire space of optimal solutions?) and more problems with people, because they often will fail to recognize that two solutions with different parameter values are really the same model. For this reason, people usually apply restrictions that create a one-to-one correspondence between models and their parameters. $\endgroup$
    – whuber
    Commented Sep 24, 2019 at 11:50
  • $\begingroup$ I think I understand what you're getting at. Since those two models I've proposed are equivalent mathematically, having different models in software could cause a problem because the software wouldn't know which parameter to choose (for example if $\beta_0$ is estimated to be $0.5$ in one model, but in the second model it's estimated to be $1.5$ (but the error term has zero mean this time)? $\endgroup$ Commented Sep 24, 2019 at 12:24
  • $\begingroup$ @DavidWetherbee now that you seem to understand the issue, why don’t you post an answer? It’s OK to answer your own question and accept your answer, and it cuts down on unanswered questions on this site. $\endgroup$
    – EdM
    Commented Sep 24, 2019 at 12:32

2 Answers 2

2
$\begingroup$

My answer:

Although it is mathematically correct to do this, a couple of problems arise when not assuming the condition that $\mathbb{E}(\epsilon|X) = 0$.

If this condition is not assumed, there is ambiguity in the estimates for the parameters. For example, if the intercept term is estimated to be $0.5$ in the first model, the (equivalent) second model can give us an estimate of the intercept term as being $1.5$ (if originally, $\mathbb{E}(\epsilon|X) = 1 \neq 0$.

$\endgroup$
0
$\begingroup$

It is correct that when an intercept term is included, the assumption $E[\epsilon|X]=0$ can be relaxed to $E[\epsilon|X]=c$, where $c$ is some constant. This is what you have derived above, and is not a violation of strict exogeneity.

Strict exogeneity is violated when the conditional expectation varies with the value of $X$, i.e. $E[\epsilon|X=x]=w(x)$. For example, consider a simple linear regression model:

$y=\beta_0+\beta_1+\epsilon$

If there is an omitted variable that (1) affects the dependent variable and (2) is correlated/varies with $X$, then $E[y|X]=\beta_0+\beta_1X+E[\epsilon|X]$. You can see that $E[\epsilon|X]$ no longer drops out of the right-hand side of the equation (as is the case when strict exogeneity holds). Note that omitted variables are not the only source of endogeneity.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.