# Doubt on derivation of OLS estimators as unbiased estimators of Optimal Linear Predictors

I'm studying from C. Shalizi's lecture notes https://www.stat.cmu.edu/~cshalizi/ADAfaEPoV/ .

In the third chapter he introduces the optimal linear estimator of a random variable $$Y$$ conditioned to another (possibly vector) $$X$$: $$f(X)=\beta X,\qquad \beta = \frac{1}{\text {Cov}(X,X)}\text {Cov}(X,Y).$$

Defining the error $$Y-f(X)=\epsilon$$ he states that, in general, $$\mathbb E(\epsilon|X)\neq 0$$, which I understand.

However, at page 45 he is proving that the Ordinary Least Squares estimators $$\hat \beta$$ give unbiased estimates of $$\beta$$ (as far as I understand, without any assumption about the actual correctness of the linear model). Here's the derivation.

My confusion concerns the step from Eq. (2.24) to (2.25), i.e. the second $$+0$$. Isn't he assuming here that the conditional expectation is $$\mathbb E (\epsilon \vert X)=0$$? And, relatedly, why in Eq. (2.24) has the $$\mathbb E(|\boldsymbol X = \boldsymbol x)$$ for $$\mathbb \epsilon$$ been replaced by an apparently unconditional expectation mean?

After some thought I realized this is probably just an error/typo from the author, which really meant that the unconditional expectation (averaged over the data set $$\boldsymbol X=\boldsymbol x$$) of the $$\hat \beta$$ estimator is equal to $$\beta$$. Indeed, it doesn't make much sense to think of being able to estimate the full regression line by making repeated measurements of $$Y$$ for few fixed values of $$X$$... unless the truth is a linear model, for sure :-)

If nobody comes up with corrections or anything to add, I will add the above as answer.

• yes. what you say is correct. formally, it seems that the author is using what is referred to as the tower property: $E(E(\epsilon | X = x)) = E(\epsilon) = 0$. May 29 '20 at 22:36

The linear coefficient estimator(s) $$\hat \beta$$ is conditionally (over the input data) unbiased only if the underlying process is truly linear.
On the other hand, $$\hat \beta$$ is an unconditionally unbiased estimator of the optimal linear predictor. This can be formally proved by integrating Eq. (2.24) over the marginal $$X$$ distribution and using the tower property as pointed out by @mlofton.