Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Zero conditional mean of the error term is one of the key conditions for the regression coefficients to be unbiased.
My question is: how can this assumption at all be violated if errors are equal to real observations of Y minus their conditional means (means for a slice of the sample described by the same value of X)?
Shouldn't the conditional expected value (for a slice of the sample described by the same value of X) of such errors always be equal to zero?
In a more technical parlance, I believe your asking, is the strict exogeneity assumption ever violated. Where the strict exogeneity assumption is...
$$E(\epsilon|X)=0$$
In practice this happens all the time. As a matter of fact the majority of the field of econometrics is focused on the failure of this assumption. When does this happen...
Let's assume that $\epsilon \sim N(0,1)$, so $E(\epsilon) = 0$. We know that if $\epsilon$ and $X$ are independent then $E(\epsilon|X) = E(\epsilon) = 0 $. However, what if $X$ and $\epsilon$ are correlated such that $Cov(X,\epsilon) = E(X'\epsilon) - E(X)E(\epsilon) = E(X'\epsilon) \neq E(\epsilon) = 0$. This implies that $E(\epsilon|X) \neq 0 $
Clearly the strict exogeneity assumption fails if $X$ and $\epsilon$ are correlated. The question is, does this ever happen? The answer is yes. As a matter of fact, outside of experimental settings, it happens more often then not. The most common example is omitted variable bias. Matthew Gunn's post discusses this. Another pedagogical example is as follows, imagine you run a regression of ice cream sales over time on the number of people wearing shorts over time. You will likely get a very large and significant parameter estimate. However you're not going to go running to Haagen Daz executives telling them they should start running advertisements for summer wear. It is obvious that there is a missing variable, temperature. This is a violation of the strict exogeneity assumption because number of people wearing shorts ($X$) is correlated with our omitted variable temperature which is contained in the error term ($\epsilon$)
Notice that the parameter estimate in our simple ice cream sales on number of shorts model is biased. Once we include the temperature in the model the, the number of shorts parameter will change. More formally:
$$\hat \beta = (X'X)^{-1}X'Y = \beta + (X'X)^{-1}X'\epsilon$$ If $X$ and $\epsilon$ are correlated then ...
$$E(\hat \beta) = \beta + (X'X)^{-1}E(X' \epsilon)$$
So the bias is $(X'X)^{-1}E(X' \epsilon)$ which vanishes if the $E(X' \epsilon)=0$
In American football, the total score is given by:
Total football score = 6 * (Touchdowns) + 1 * (ExtraPoints) + 2 * (TwoPointConversions) + 2 * (safeties) + 3 * field goals.
But if you ran the regression:
TotalFootBallScore = b1 * touchdowns + b2 * fieldgoals + e
You wouldn't estimate a value of 6 for b1. Regress the total football score on number of touchdowns and field goals, and you would almost certainly estimate that touchdowns are worth more 7 or more points rather than 6. Your error term e in this case contains the points scored from extra points and two points conversions, and those are almost certainly not zero conditional on knowing the number of touchdowns.
