# The correct condition for OLS estimates to be unbiased?

For the ordinary least square (OLS) estimates of regression ($$\vec{y} =\mathbf{X} \cdot \vec{\beta} + \vec{\epsilon}$$) to be unbiased (without considering the efficiency), which one of the three conditions is required (sufficient):

$$\mathbb{E}(\vec{\epsilon}) = \vec{0} \tag{Assumption 1}$$

$$\mathbb{E}(\vec{\epsilon} | \mathbf{X}) = \vec{0} \tag{Assumption 2}$$

$$\mathbb{E}(\mathbf{X}^T \cdot \vec{\epsilon}) = \vec{0} \tag{Assumption 3}$$

I came to the first one when deriving by myself:

The expectation of $$\hat{\vec{\beta}}$$ is:

\begin{align*} \mathbb{E}(\hat{\vec{\beta}}) & = \mathbb{E}[\vec{\beta} + (\mathbf{X}^T \cdot \mathbf{X})^{-1} \cdot \mathbf{X}^T \cdot\vec{\epsilon}] \\ & = \mathbb{E}(\vec{\beta}) + \mathbb{E}[(\mathbf{X}^T \cdot \mathbf{X})^{-1} \cdot \mathbf{X}^T \cdot\vec{\epsilon}] \\ & as\ \vec{\beta} \ is \ constant\ and\ \mathbf{X}\ is\ also\ constant\\ & = \vec{\beta} + (\mathbf{X}^T \cdot \mathbf{X})^{-1} \cdot \mathbf{X}^T \cdot \mathbb{E}(\vec{\epsilon}) \\ \end{align*}

But some materials in econometrics stated the Assumption 2 or Assumption 3. I am confused as $$\mathbf{X}$$ is considered observed or fixed in OLS, conditioned expectation in Assumption 2 doesn't seem to make sense. And Assumption 3 should easily be simplified to Assumption 1 given $$\mathbf{X}$$ fixed.

If you regard $$X$$ as fixed then assumptions 1 and 2 are the same. Since $$X$$ often isn't fixed, just conditioned on, it's helpful to have the assumption in a form that doesn't rely on $$X$$ being fixed. That's the point of assumption 2. Both assumptions 1 and 2 are saying that $$E[Y|X=x]$$ is truly is $$x\beta$$.
For example, suppose $$E[Y|X=x]$$ is actually a curve, so that $$E[Y|X=x]\neq x\beta$$. There will still be a best-fitting true line, a value of $$\beta$$ that makes $$E[X^Te]=0$$, but we will not have $$E[e_i|X_i=x]=0$$ for any individual point. In this case it is still true that $$\hat\beta$$ is unbiased for the slope of the true best-fitting line, but that line is not $$E[Y|X=x]$$.
• Thx! This makes so much sense. But if we assume $\mathbf{X}$ to be random too: \begin{align*} \mathbb{E}(\hat{\vec{\beta}}) = \vec{\beta} + \mathbb{E}_{X}[(\mathbf{X}^T \cdot \mathbf{X})^{-1} \cdot \mathbf{X}^T \cdot\mathbb{E}_{\vec{\epsilon}|X}(\vec{\epsilon}|\mathbf{X})] \end{align*} When $\mathbb{E}_{\vec{\epsilon}|X}(\vec{\epsilon}|\mathbf{X}) = 0$, it is indeed unbiased (sufficient). But this isn't a necessary condition (non-zero inner expectations can cancel out in the outer one), right? Sep 12, 2023 at 19:34
• Thanks a gain Thomas! I don't quite get how to derive assumption 3 when $\mathbf{X}$ is random. As $(\mathbf{X}^T \cdot \mathbf{X})^{-1}$ in $\mathbb{E}[(\mathbf{X}^T \cdot \mathbf{X})^{-1} \cdot \mathbf{X}^T \cdot\vec{\epsilon}]$ is also random, it seems we cannot take it out directly. How can we prove that $\mathbb{E}(\mathbf{X}^T \cdot \vec{\epsilon}) = \vec{0}$ will satisfy the unbiasness? Sep 13, 2023 at 0:51
• Condition on $X$. It's unbiased for every fixed $X$ and therefore it's unbiased for random $X$. Sep 13, 2023 at 2:22