Zero conditional expectation of error in OLS regression

Question

Suppose we have a dependent variable $Y$ and an independent variable $X$ in a population, and we want to estimate the linear model $$ Y = \beta_{0} + \beta_{1}X + \varepsilon $$ Using the least-squares method, we obtain estimates $\hat{\beta_{0}}$ and $\hat{\beta_{1}}$, and so in a sample of this population, we have for each $i$ in the sample $$ y_{i} = \hat{\beta_{0}} + \hat{\beta_{1}}x_{i} + e_{i} $$ where $e_{i}$ is the residual associated with observation $i$. Now, one essential assumption here is that the conditional distribution of $e_{i}$ given an $X$ is normal, and $$ \mathbb{E}(e_{i}|X) = 0 $$ I don't fully understand how $e_{i}$ can be looked at as a random variable given an $X$. What precisely is the random variable $e_{i}$, i.e. what different values can it take on? Given estimates $\hat{\beta_{0}}$ and $\hat{\beta_{1}}$ and a value $X$, it seems to me that the $e_{i}$ just take on a finite number of fixed values (could even be 1); so in what sense is it looked at as a random variable?

Alternatively, does the "randomness" in $e_{i}$ come because we consider the error terms associated with different estimates of the regression coefficients? In other words, does the zero conditional expectation of errors mean that given an $X = x$, if we picked different samples of the population containing $x$ and estimated the least squares line for each of these samples, the error associated with $x$ should, on average, be zero?

Answer 1

Residuals, defined given the regressors, remain random variables simply because, even if the regressors are given, is not possible to reduce them to constants. In other words if you have $x_i$ you can obtain, given estimated coefficients, the predicted values of $y$ but this prediction maintain its uncertainty.

However you have right that the residual values are linked to the estimated coefficients.

Now you have to note that the condition you wrote $E[e_i|X]=0$ is wrong because is written on residuals. I fear that you conflate the meaning of residuals and errors. This problem is widely spread and very dangerous.

Following your notation the condition should be $E[\epsilon_i|X]=0$ and its make sense only if we interpret the true model as structural equation and not as something like population regression (you speak about linear model in your question, too general and ambiguous name frequently used). Misunderstanding like those have produced many problems among students and in literature also.

Those posts can help you and other readers:

What is the actual definition of endogeneity?

Does homoscedasticity imply that the regressor variables and the errors are uncorrelated?

Endogeneity testing using correlation test

Regression's population parameters

Answer 2

Some of the confusion regards difference between $e$ and $\epsilon$, and that seems to have been addressed adequately in the comments and other answer(s). But additional confusion expressed by the OP concerns the nature of randomness itself in this context, and in the related issue of the meaning of $E(\epsilon | X)$. Here is an answer that clarifies these issues.

Consider a classic example: $Y$ = son's adult height, $X$ = father's adult height. Suppose $E(Y | X = x) = \beta_0 + \beta_1 x$ is true. Since this is a model for how data might appear, we need some conceptual framework for where/when/how the data are collected. Suppose, for the sake of concreteness, that we are talking about a "typical" sample of people living in the world today, one that is reasonably representative of this human spectrum.

The question of "randomness" can be best understood as something that is unrelated to the actual data; which instead can be understood in terms of "potentially observable data" for the conceptual data collection framework. Given a particular father whose height is 180 cm, but who is otherwise generic within the sampling framework, there is a distribution of potentially observable son's heights. Thus the $Y$ in the expression $Y | X = 180$ can be described as "random" at this stage, having some probability distribution of potentially observable values.

(Note that the "population" of the world is irrelevant in this context - instead, the regression model views the heights of people in the world today as themselves but one of many possible realization of possible heights that could have existed at this particular point in time. One reason the "population" framework makes no sense is that there is no data in population from which to construct the population conditional distributions: How many fathers on the planet have height between 79.9999999...........9 and 80.0000..........1 centimeters? The answer is "none" if you let the "..." run on long enough.)

Now, $\epsilon = Y - (\beta_0 + \beta_1 x)$, which is the difference between the potentially observable (random) $Y$ and the mean of the distribution of such potentially observable $Y$ for the given $x$. The "randomness" in $\epsilon$ is inherited from the "randomness" in $Y$ ( the conditional mean $\beta_0 + \beta_1 x$, while uncertain in the mind, is scientifically fixed in this context).

To understand the condition $E(\epsilon | X=x) = 0$, consider again $X=180$. Here, $\epsilon$ is the deviation of a potentially observable $Y$ for which $X=180$, from the mean of all such potentially observable $Y$. The mean of all such $\epsilon$'s is 0 precisely because the mean of all such $Y$'s is $\beta_0 + \beta_1 (180)$.

By the way, the assumption $E(\epsilon | X=x) = 0 $ is not needed here: it is a mathematical consequence of the more intuitive assumption $E(Y | X = x) = \beta_0 + \beta_1 x$, which simply states that the regression mean function is correctly modeled.