My question is not particularly straightforward. I will explain my reasoning, and then give the question at the end to avoid any confusion.
$\ Y_{i} = \beta_{0}+\beta_{1}X_{i}+\varepsilon_{i} $
Suppose that this our population regression function. We note that $\ \beta_{0}$ and $\ \beta_{1} $ are not arbitrary. This means that the error term $\ \varepsilon_{i} $ has some structure to it. In particular, it reflects the deviation of $ Y_{i} $ from its expected value. To put it even more clearly, we have that $ E[Y_{i}|X_{i}]= \beta_{0}+\beta_{1}X_{i} $. What I am trying to say here is that our population parameters must take some non-arbitrary value, otherwise we might as well define our errors to make the equation balance and pick them randomly.
Now suppose we want to take a sample and estimate the above population regression function. But before doing so, we want to work out what the true value of $ \beta_{0} $ and $ \beta_{1} $ are. To do this we appeal to the mean square error as our guide (I drop the subscripts for simplicity of notation):
$ MSE(b_{0}, b_{1}) = E[(Y - (b_{0}+b_{1}X))^2] = E[Y^2] -2b_{0}E[Y]-2b_{1}E[XY]+E[(b_{0}+b_{1}X)^2]$
$ MSE(b_{0}, b_{1}) = E[Y^2] -2b_{0}E[Y]-2b_{1}Cov(X,Y) - 2b_{1}E[X]E[Y]+b_{0}^2+2b_{0}b_{1}E[X]+b_{1}^2Var[X]+b_{1}^2(E[X])^2$
I have skipped some of the algebra because it is not really important at this point. Now I take first order conditions with respect to my two variables:
$ \frac{\partial MSE(b_{0},b_{1})}{\partial b_0}= -2E[Y]+2b_{0}+2b_{1}E[X]$
$ \frac{\partial MSE(b_{0},b_{1})}{\partial b_1}= -2Cov(X,Y)-2E[X]E[Y]+2b_{0}E[X]+2b_{1}Var[X]+2b_{1}(E[X])^2 $
Setting the partial derivates equal to zero and solving for each parameter gives:
$ \beta_{0}=E[Y]-\beta_{1}E[X] $
$ \beta_{1}= \frac{Cov[X,Y]}{Var[X]} $
Now that we have the equations of our true population parameters, we can consider the issue at hand, otherwise known as my question! Let's try to check the correlation between our explanatory variables and the error term defined above:
$\ E[X\varepsilon] = E[X(Y-(\beta_{0}+\beta_{1}X)] = E[XY - X\beta_{0}-\beta_{1}X^2] $
Using some simple algebraic manipulations we get that:
$ E[X\varepsilon]= E[XY]- E[X\beta_{0}+\beta_{1}X^2] $
$ E[X\varepsilon]= E[XY]- E[X(E[Y]-\beta_{1}E[X])+\beta_{1}X^2] $
$ E[X\varepsilon]= E[XY]- E[XE[Y]-X\beta_{1}E[X]+\beta_{1}X^2] $
$ E[X\varepsilon]= E[XY]- E[Y]E[X] + E[X\beta_{1}E[X]] - E[\beta_{1}X^2] $
$ E[X\varepsilon]= E[XY]- E[Y]E[X] + \beta_{1}E[X]^2 - E[\beta_{1}X^2] $
$ E[X\varepsilon]= E[XY]- E[Y]E[X] + \beta_{1}E[X]^2 - \beta_{1}E[X^2] $
$ E[X\varepsilon]= E[XY]- E[Y]E[X] -\beta_{1}(E[X^2] - E[X]^2) $
$ E[X\varepsilon]= Cov[X,Y]-\beta_{1}Var[X] $
$ E[X\varepsilon]= Cov[X,Y]-\frac{Cov[X,Y]}{Var[X]}Var[X] $
$ E[X\varepsilon]= 0 $
Now what we have proved is the by construction the parameters of the population regression function are defined in such a way that the covariance between the explanatory variables and errors is equal to zero.
Now bearing all of the above in mind, herein lies my question: how is endogeneity mathematically possible? Is it that my interpretation of the regression is wrong? I have heard others mention that there is some sort of causal interpretation of regression that must be taken, though I am not really sure what that means, or what that is. Any insights?
P.S. Thank you to those that have answered my other questions about regression analysis previously, I am learning a lot!