Skip to main content
added 2 characters in body
Source Link
User1865345
  • 10.3k
  • 12
  • 23
  • 40

This is a great example that illustrates why, in regression models, (i) the assumption $E(u | x) = 0$ should not be used, and (ii) the "population" framework should not be used.

Rather than the assumption $E(u | x) =0$, it would make much more sense to state the assumption in the equivalent form $E(y | X=x) = \beta_0 +\beta_1 x$. This assumption states that the means of the conditional distributions fall exactly on a line of the form $\beta_0 + \beta_1 x$, for some $\beta_0$, $\beta_1$.

As the OP notes, the conditional distributions in the population framework are all degenerate, so that the mean is just equal to the single $y$ value. For example, the distribution of $Y | X = 9$ is given by $Pr(Y = 20 | X=9) = 1$$\Pr(Y = 20 | X=9) = 1$, with $Pr(Y = y | X=9) = 0$$\Pr(Y = y | X=9) = 0$, for all $y \neq 20$. The mean of this distribution is clearly 20.

Since these conditional mean values do not all fall precisely on a straight line, the assumption $E(y | X=x) = \beta_0 +\beta_1 x$ is violated. This explains the OP's finding that $E(u | x) \neq 0$.

Here is an example "population" where it works.

$$\begin{array}{c|c|c|} & \text{x} & \text{y} \\ \hline \text{} & 1 & 12 \\ \hline \text{} & 1 & 14 \\ \hline \text{} & 1 & 16\\ \hline \text{} & 2 & 20\\ \hline \text{} & 2 & 24\\ \hline \text{} & 2 & 28\\ \hline \text{} & 3 & 31\\ \hline \text{} & 3 & 33\\ \hline \text{} & 3 & 38\\ \hline \end{array}$$

Here the conditional means are 14, 24, 34, falling on a linear function of $x = 1,2,3$. Consider the distribution of $y | X=3$:

$$\begin{array}{c|c|c|} & \text{p(y|x)} & \text{y} \\ \hline \text{} & 1/3 & 31\\ \hline \text{} & 1/3 & 33\\ \hline \text{} & 1/3 & 38\\ \hline \end{array}$$

The distribution of $u$ is obtained by replacing the $y$ values with $y-34$, so $E(u | X = 3) = (1/3)(31 - 34) + (1/3)(33-34) + (1/3)(38-34) = 0.$

If the means of the distributions are configured so that they do not fall exactly on a line, then $E(u | X = x) \neq 0$ for some $x$.

This example also illustrates the point that the "population model" should not be used to define the regression model. As the examples illustrate, the conditional means are not really true means in the scientific, generalizable sense, they are instead quite noisy due to small sample sizes in the subpopulation defined by the "$| x$." In some cases, there may be no observations whatsoever in such subsets of the population, even when the population is large. This problem is magnified multiplicatively in the case of multiple regression.

This is a great example that illustrates why, in regression models, (i) the assumption $E(u | x) = 0$ should not be used, and (ii) the "population" framework should not be used.

Rather than the assumption $E(u | x) =0$, it would make much more sense to state the assumption in the equivalent form $E(y | X=x) = \beta_0 +\beta_1 x$. This assumption states that the means of the conditional distributions fall exactly on a line of the form $\beta_0 + \beta_1 x$, for some $\beta_0$, $\beta_1$.

As the OP notes, the conditional distributions in the population framework are all degenerate, so that the mean is just equal to the single $y$ value. For example, the distribution of $Y | X = 9$ is given by $Pr(Y = 20 | X=9) = 1$, with $Pr(Y = y | X=9) = 0$, for all $y \neq 20$. The mean of this distribution is clearly 20.

Since these conditional mean values do not all fall precisely on a straight line, the assumption $E(y | X=x) = \beta_0 +\beta_1 x$ is violated. This explains the OP's finding that $E(u | x) \neq 0$.

Here is an example "population" where it works.

$$\begin{array}{c|c|c|} & \text{x} & \text{y} \\ \hline \text{} & 1 & 12 \\ \hline \text{} & 1 & 14 \\ \hline \text{} & 1 & 16\\ \hline \text{} & 2 & 20\\ \hline \text{} & 2 & 24\\ \hline \text{} & 2 & 28\\ \hline \text{} & 3 & 31\\ \hline \text{} & 3 & 33\\ \hline \text{} & 3 & 38\\ \hline \end{array}$$

Here the conditional means are 14, 24, 34, falling on a linear function of $x = 1,2,3$. Consider the distribution of $y | X=3$:

$$\begin{array}{c|c|c|} & \text{p(y|x)} & \text{y} \\ \hline \text{} & 1/3 & 31\\ \hline \text{} & 1/3 & 33\\ \hline \text{} & 1/3 & 38\\ \hline \end{array}$$

The distribution of $u$ is obtained by replacing the $y$ values with $y-34$, so $E(u | X = 3) = (1/3)(31 - 34) + (1/3)(33-34) + (1/3)(38-34) = 0.$

If the means of the distributions are configured so that they do not fall exactly on a line, then $E(u | X = x) \neq 0$ for some $x$.

This example also illustrates the point that the "population model" should not be used to define the regression model. As the examples illustrate, the conditional means are not really true means in the scientific, generalizable sense, they are instead quite noisy due to small sample sizes in the subpopulation defined by the "$| x$." In some cases, there may be no observations whatsoever in such subsets of the population, even when the population is large. This problem is magnified multiplicatively in the case of multiple regression.

This is a great example that illustrates why, in regression models, (i) the assumption $E(u | x) = 0$ should not be used, and (ii) the "population" framework should not be used.

Rather than the assumption $E(u | x) =0$, it would make much more sense to state the assumption in the equivalent form $E(y | X=x) = \beta_0 +\beta_1 x$. This assumption states that the means of the conditional distributions fall exactly on a line of the form $\beta_0 + \beta_1 x$, for some $\beta_0$, $\beta_1$.

As the OP notes, the conditional distributions in the population framework are all degenerate, so that the mean is just equal to the single $y$ value. For example, the distribution of $Y | X = 9$ is given by $\Pr(Y = 20 | X=9) = 1$, with $\Pr(Y = y | X=9) = 0$, for all $y \neq 20$. The mean of this distribution is clearly 20.

Since these conditional mean values do not all fall precisely on a straight line, the assumption $E(y | X=x) = \beta_0 +\beta_1 x$ is violated. This explains the OP's finding that $E(u | x) \neq 0$.

Here is an example "population" where it works.

$$\begin{array}{c|c|c|} & \text{x} & \text{y} \\ \hline \text{} & 1 & 12 \\ \hline \text{} & 1 & 14 \\ \hline \text{} & 1 & 16\\ \hline \text{} & 2 & 20\\ \hline \text{} & 2 & 24\\ \hline \text{} & 2 & 28\\ \hline \text{} & 3 & 31\\ \hline \text{} & 3 & 33\\ \hline \text{} & 3 & 38\\ \hline \end{array}$$

Here the conditional means are 14, 24, 34, falling on a linear function of $x = 1,2,3$. Consider the distribution of $y | X=3$:

$$\begin{array}{c|c|c|} & \text{p(y|x)} & \text{y} \\ \hline \text{} & 1/3 & 31\\ \hline \text{} & 1/3 & 33\\ \hline \text{} & 1/3 & 38\\ \hline \end{array}$$

The distribution of $u$ is obtained by replacing the $y$ values with $y-34$, so $E(u | X = 3) = (1/3)(31 - 34) + (1/3)(33-34) + (1/3)(38-34) = 0.$

If the means of the distributions are configured so that they do not fall exactly on a line, then $E(u | X = x) \neq 0$ for some $x$.

This example also illustrates the point that the "population model" should not be used to define the regression model. As the examples illustrate, the conditional means are not really true means in the scientific, generalizable sense, they are instead quite noisy due to small sample sizes in the subpopulation defined by the "$| x$." In some cases, there may be no observations whatsoever in such subsets of the population, even when the population is large. This problem is magnified multiplicatively in the case of multiple regression.

Source Link
BigBendRegion
  • 6k
  • 1
  • 20
  • 35

This is a great example that illustrates why, in regression models, (i) the assumption $E(u | x) = 0$ should not be used, and (ii) the "population" framework should not be used.

Rather than the assumption $E(u | x) =0$, it would make much more sense to state the assumption in the equivalent form $E(y | X=x) = \beta_0 +\beta_1 x$. This assumption states that the means of the conditional distributions fall exactly on a line of the form $\beta_0 + \beta_1 x$, for some $\beta_0$, $\beta_1$.

As the OP notes, the conditional distributions in the population framework are all degenerate, so that the mean is just equal to the single $y$ value. For example, the distribution of $Y | X = 9$ is given by $Pr(Y = 20 | X=9) = 1$, with $Pr(Y = y | X=9) = 0$, for all $y \neq 20$. The mean of this distribution is clearly 20.

Since these conditional mean values do not all fall precisely on a straight line, the assumption $E(y | X=x) = \beta_0 +\beta_1 x$ is violated. This explains the OP's finding that $E(u | x) \neq 0$.

Here is an example "population" where it works.

$$\begin{array}{c|c|c|} & \text{x} & \text{y} \\ \hline \text{} & 1 & 12 \\ \hline \text{} & 1 & 14 \\ \hline \text{} & 1 & 16\\ \hline \text{} & 2 & 20\\ \hline \text{} & 2 & 24\\ \hline \text{} & 2 & 28\\ \hline \text{} & 3 & 31\\ \hline \text{} & 3 & 33\\ \hline \text{} & 3 & 38\\ \hline \end{array}$$

Here the conditional means are 14, 24, 34, falling on a linear function of $x = 1,2,3$. Consider the distribution of $y | X=3$:

$$\begin{array}{c|c|c|} & \text{p(y|x)} & \text{y} \\ \hline \text{} & 1/3 & 31\\ \hline \text{} & 1/3 & 33\\ \hline \text{} & 1/3 & 38\\ \hline \end{array}$$

The distribution of $u$ is obtained by replacing the $y$ values with $y-34$, so $E(u | X = 3) = (1/3)(31 - 34) + (1/3)(33-34) + (1/3)(38-34) = 0.$

If the means of the distributions are configured so that they do not fall exactly on a line, then $E(u | X = x) \neq 0$ for some $x$.

This example also illustrates the point that the "population model" should not be used to define the regression model. As the examples illustrate, the conditional means are not really true means in the scientific, generalizable sense, they are instead quite noisy due to small sample sizes in the subpopulation defined by the "$| x$." In some cases, there may be no observations whatsoever in such subsets of the population, even when the population is large. This problem is magnified multiplicatively in the case of multiple regression.