# Is it possible for the zero conditional mean assumption to fail?

I have a questions about the so-called "zero conditional mean" assumption often made in the context of regression analysis. I am struggling to see how it could be violated, or rather where it is violated.

$$E[\hat \beta] = E[\left(\mathbf X' \mathbf X\right)^{-1}\mathbf X' \mathbf Y] = \beta + E[\left(\mathbf X' \mathbf X\right)^{-1}\mathbf X' \mathbf \varepsilon]$$

Above, I give the expectation of the OLS estimate in matrix form. My problem is that I do not see how it is possible for $$\ E[X' \varepsilon]$$ to not equal zero. Now before you all give me examples from the literature on endogeneity of when there is this bias, the reason I have trouble with this is because whenever we formulate our population regression function as a conditional expectation, by definition the $$E[\ X' \varepsilon] =0$$. In other words, in any population regression function defined as a conditional expectation, we have that the errors are uncorrelated with our regressors. This is because their correlation is defined to be zero. To put it formally (and by the Law of Iterated Expectations):

$$\ E[Y|X]=B_{0}+B_{1}X_{i} \Rightarrow Y_{i}=E[Y|X]+\varepsilon _{i}\Rightarrow E[Y|X_{i}]=E[Y|X]+E[\varepsilon|X ]\Rightarrow E[\varepsilon_{i}|X]=0\$$

But this raises the question: what is the correlation term in the OLS estimates referring to, and how can it be that the assumption does not hold?

EDIT: Some of the comments have suggested that I am confusing errors with residuals. However, this is not obviously the case. My point is as follows. When running a regression, we use OLS to tell us about: $$\ E[Y|X]$$. However, for any way in which I specify $$\ E[Y|X]$$ it seems as though I am mathematically committing myself to $$\ E[\varepsilon_{i}|X]=0\$$. Could someone perhaps specify a population regression function (as a conditional expectation) where this is not the case?

For example, if we consider $$\ E[Y|X] = B_{0}+B_{1}X_{i}+B_{2}X_{i}$$ and we suppose that $$E[\varepsilon_{i}|X] = g(x)$$. Then, by iterated expectations we get that:

$$\ Y_{i}=B_{0}+B_{1}X_{i}+B_{2}X_{i}+\varepsilon_{i}$$

$$\ E[Y|X]=E[B_{0}+B_{1}X_{i}+B_{2}X_{i}|X] + E[\varepsilon|X]$$

$$\ E[Y|X]=E[Y|X] + E[\varepsilon|X]$$

$$\ E[Y|X]=E[Y|X] +g(x)$$

$$\ g(x)=0$$

If $$\ g(x)$$ does not equal zero, then we have a mathematical contradiction, and therefore I have no idea how in the OLS estimates, we can have anything other than $$\ E[X' \varepsilon=0]\$$.

EDIT 2:

It seems I have found a potential solution to my problem:

CEF: $$\ E[Y|X]=B_{0}+B_{1}X_{i}+B_{2}X_{i}$$

Regression $$\ Y_{i}= B_{0}+B_{1}X_{i}+B_{2}X_{i} + \varepsilon_{i}$$

The idea is that $$\ E[X|\varepsilon]=0$$ must hold for the CEF, but it may not hold for the regression function. Is this on the right track?

• Could you please explain what you mean by "$u$"? This might be the key to the question.
– whuber
Commented Oct 29, 2021 at 15:23
• I am sorry, that is a typo! I would think that the u above ought to be the same as the epsilon below, since both refer to the population and not the residuals. I'll change it! Commented Oct 29, 2021 at 15:26
• Thank you. But now I cannot follow what you might mean by "I give the expectation of the OLS estimate," because no expectations are involved in your formula for $\hat \beta.$
– whuber
Commented Oct 29, 2021 at 15:29
• Sigh I am really bad at this. Now that should fix it. Commented Oct 29, 2021 at 15:33
• You are, possibly, confusing the residuals with the errors. The residuals are the deviations of the observations from the estimates, and it is true that, by construction, they are uncorrelated with $X$. The errors are the deviations of the observations from $\beta X$, and they can be correlated with $X$. Simply assuming that they are not doesn't make it so! Commented Oct 29, 2021 at 15:33

The short answer is yes, $$\mathbb{E}[X_i\varepsilon_i] = 0$$ by construction.

A few important remarks:

• However, this does not imply that $$\mathbb{E}[\varepsilon_i \mid X_i] = 0$$.
• The conditional mean equal to zero is a much stronger condition.
• Your logic fails because you are assuming that conditional means are always linear.
• Statistically the OLS $$\beta$$ always satisfies $$\mathbb{E}[X_i\varepsilon_i] = 0$$.
• Modern theory calls this a "pseudo-parameter", that "mechanically" satisfies the restriction.
• There are many examples where we can formally define an external notion of a "true" $$\beta^*$$, and show that things like sample selection, missing variables, etc., can lead to situations where $$\beta \ne \beta^*$$.This requires more structure to the problem to even define the bias.

The way that you are looking at the problem is slightly circular. Let us break down the problem by looking at some definitions.

Setting up the problem

The OLS estimator in matrix form is $$\beta = (X'X)^{-1}X'Y$$. It can also be written as follows: $$\hat{\beta} = \left( \frac{1}{n} \sum_{i=1}^n X_i X_i'\right)^{-1}\left( \frac{1}{n} \sum_{i=1}^n X_i Y_i \right)$$ The OLS is an approximation to the following population quantity $$\beta = \mathbb{E}[X_iX_i']^{-1}\mathbb{E}[X_iY_i]$$ This definition is convenient because it doesn't introduce any assumptions about the error terms. It just says that $$\beta$$ is a function of two means.

Why $$\mathbb{E}[X_i\varepsilon_i] = 0$$ by construction?

Now let us define the error terms as $$\varepsilon_i = Y_i - X_i'\beta$$ Then we can verify that \begin{align*} \mathbb{E}[X_i\varepsilon_i] &= \mathbb{E}[X_iY_i-X_iX_i' \beta] \\ &=\mathbb{E}[X_iY_i] - \mathbb{E}[X_iX_i'] \beta \end{align*} Plug-in the definition of $$\beta$$ above and you can easily check that $$\mathbb{E}[X_i\varepsilon_i] = 0$$. A similar result holds in finite sample for $$\hat{\beta}$$. In particular $$\frac{1}{n}\sum_{i=1}^2 X_i(Y_i - X_i'\hat{\beta}) = \left(\frac{1}{n}\sum_{i=1}^2 X_iY_i\right) - \left(\frac{1}{n}\sum_{i=1}^2 X_iX_i' \right)\hat{\beta}$$ This can also be written in matrix form as $$\hat{\varepsilon} = Y - X\hat{\beta}$$, and $$\frac{X'\hat{\varepsilon}}{n} = 0$$.

Why $$\mathbb{E}[\varepsilon_i \mid X_i]$$ may not equal zero?

So far we have not made any assumptions about the dependence between $$(Y_i,X_i)$$. Now consider a general model where $$Y_i = g(X_i) + U_i, \qquad \mathbb{E}[U_i \mid X] = 0$$ where $$U_i$$ is a new variable that we are introducing. The function $$g(X_i)$$ is the conditional mean of $$Y$$. Now let us apply the definition of the OLS residual to this model: \begin{align*} \varepsilon_i &= Y_i - X_i'\beta \\ &= g(X_i) + U_i - X_i'\beta \\ &= [g(X_i) - X_i'\beta] + U_i \end{align*} Now let us compute the conditional expectation \begin{align*} \mathbb{E}[\varepsilon_i \mid X_i] &= \mathbb{E}[g(X_i) - X_i'\beta \mid X_i] - \mathbb{E}[U_i \mid X_i] \\ &= g(X_i) - X_i'\beta \end{align*} In the second line we use the fact that the conditional mean of $$U_i$$ is zero, and $$g(X_i) - X_i'\beta$$ is a function of $$X_i$$. Once you state it in this form it is easy to find counterexamples of when the conditional mean is non-zero, regardless of the value of $$\beta$$. For instance, assume that $$g(X_i) = X_i^2$$ and $$X_i \sim \mathcal{N}(0,1)$$.

• In this particular case, we can prove that the OLS coefficient is defined as $$\beta = \mathbb{E}[X_iX_i']^{-1}\mathbb{E}[X_ig(X_i)]$$, by applying the law of iterated expectations.
• As you can see, everything is internally consistent.
• In a nonlinear model like this, $$X_i'\beta$$ is called the "best linear projection" because it minimizes the residuals under the linearity constraint.
• Thank you, I think I am beginning to get the hang of this. However I have a few follow-up questions. (1) If we restrict our attention to linear models, then since the CEF is the same as the population regression function do we ever see the assumption fail? (2) If so, would this mean that the population function is mis-specified in some way, and no longer is the "best estimate" for the CEF? (3) When you gave the OLS estimates, do you not assume that E[X'e]=0 in the population? (4) Should we make a distinction between the conditional expectation function and regression function? Commented Oct 29, 2021 at 21:00
• (1) If the CEF is linear, then yes, the conditional expectation is zero. (2) Yes, correct, if the model is nonlinear, then the regression is no longer the best estimate. (3) No, I am just defining it based on the moments of $(X,Y)$, since $\beta$ in population is the solution to $\arg \max E[(Y_i - X_i'\beta)^2]$. However, as the post shows, you get $E[X_i\varepsilon_i]$ as a consequence. (4) Yes, that's the point. Commented Oct 29, 2021 at 21:58
• Cool, thank you very much! Commented Oct 29, 2021 at 22:07
• Just as another quick question, if $\ E[X_{i}\varepsilon_{i}]$ is true by construction (that is to make my population match up to the linear CEF, Beta is defined in such a way that covariates do not correlate with the errors), where would endogeneity come into the picture? Clearly the CEF stays the same, nothing wrong there. This must mean that my population regression function is somehow going wrong. Commented Oct 30, 2021 at 13:22
• Endogeneity represent the most relevant conflation between statistical and causal concepts. This fact brings readers in any sort of confusion/ambiguity. Unfortunately most Econometrics books are unreliable about that, therefore are unreliable at all. I wrote something about that. You can find most things here stats.stackexchange.com/questions/493211/… and in link therein. Commented Nov 3, 2021 at 20:20

The zero conditional mean assumption for the error term is equivalent to assuming that we have specified the correct form for the regression function in our model. The reason you are unable to see a situation where this would not hold is that your entire analysis is predicated on correct specification of the initial model. In particular, your moment results for the OLS estimator are predicated on the initial model form in the regression being correct.

In order to show when the zero conditional mean assumption does not hold, let's generalise by noting that we specify some assumed form for the regression model, but we might get this wrong ---i.e., we will look at model misspecification error. Remember that the error terms in the regression model are deviations of the response value from its true expected value under the assumed form for the true regression function. We take some assumed form for the regression function $$u_\boldsymbol{\beta}$$ and we assume that there is some "true" parameter value $$\boldsymbol{\beta}$$ such that $$u_\boldsymbol{\beta}(\mathbf{x}_i) = \mathbb{E}(Y_i|\mathbf{x}_i)$$ for all $$\mathbf{x}_i$$. We then define the error terms as deviations $$\varepsilon_i \equiv Y_i - u_\boldsymbol{\beta}(\mathbf{x}_i)$$, which gives us the assumed ​model form:

$$Y_i = u_\boldsymbol{\beta}(\mathbf{x}_i) + \varepsilon_i.$$

Now, our stipulated function $$u_\boldsymbol{\beta}$$ is designed to model the expected response as a function of the regressors. But of course, there is no guarantee of this; maybe we specified the wrong kind of function (e.g., we used a linear regression function when some kind of nonlinear model is needed).

Now, suppose we let $$u_*(\mathbf{x}_i) \equiv \mathbb{E}(Y_i|\mathbf{x}_i)$$ denote the true regression function (i.e., the true conditional expectation of the response variable). Perhaps our model is correctly specified (i.e., there exists a parameter value $$\boldsymbol{\beta}$$ giving $$u_* = u_\boldsymbol{\beta}$$) and perhaps it is not. Irrespective of this, the true conditional expected value of the error terms in the model will be:

\begin{align} \mathbb{E}(\varepsilon_i|\mathbf{x}_i) &= \mathbb{E}(Y_i - u_\boldsymbol{\beta}(\mathbf{x}_i)|\mathbf{x}_i) \\[6pt] &= \mathbb{E}(Y_i|\mathbf{x}_i) - u_\boldsymbol{\beta}(\mathbf{x}_i) \\[6pt] &= u_*(\mathbf{x}_i) - u_\boldsymbol{\beta}(\mathbf{x}_i). \\[6pt] \end{align}

From this result, we can see that if the form for our regression model is correctly specified (i.e., we have $$u_* = u_\boldsymbol{\beta}$$ under the true parameter value) then the conditional expected value of the error term is zero. Contrarily, if the form of our model is incorrectly specified (i.e., we have $$u_* \neq u_\boldsymbol{\beta}$$ under every allowable parameter value) then the conditional expectation of the error term is not generally going to be zero. (It will still be zero at specific points where the regression functions cross).

As you can see, the failure of the conditional expectation assumption reflects the fact that our assumed regression function does not excapsulate the true conditional expectation of the response variable. This problem will show up when you do the diagnostic plots for your regression analysis --- specifically, it shows up in the residual plot when you see that the residuals are not distributed randomly around the zero line.

One final thing to note here is that you are never going to be able to see this problem if you work solely within the assumed model form. If you look at the model under the assumption that $$\mathbb{E}(Y_i|\mathbf{x}_i) = u_\boldsymbol{\beta}(\mathbf{x}_i)$$ then you get zero conditional expectation for the error term, since the error term is defined as the deviation of the response variable from the assumed regression function.

• Would the following be correct: the Conditional Expectations Function is approximated (or in the linear case, equal to) the regression function. However if we have misspecified our regression function it is a bad approximation (or in the linear case, is not equal to) if the CEF. Commented Oct 30, 2021 at 10:33