0
$\begingroup$

For the ordinary least square (OLS) estimates of regression ($\vec{y} =\mathbf{X} \cdot \vec{\beta} + \vec{\epsilon}$) to be unbiased (without considering the efficiency), which one of the three conditions is required (sufficient):

$$\mathbb{E}(\vec{\epsilon}) = \vec{0} \tag{Assumption 1}$$

$$\mathbb{E}(\vec{\epsilon} | \mathbf{X}) = \vec{0} \tag{Assumption 2}$$

$$\mathbb{E}(\mathbf{X}^T \cdot \vec{\epsilon}) = \vec{0} \tag{Assumption 3}$$

I came to the first one when deriving by myself:

The expectation of $\hat{\vec{\beta}}$ is:

$$ \begin{align*} \mathbb{E}(\hat{\vec{\beta}}) & = \mathbb{E}[\vec{\beta} + (\mathbf{X}^T \cdot \mathbf{X})^{-1} \cdot \mathbf{X}^T \cdot\vec{\epsilon}] \\ & = \mathbb{E}(\vec{\beta}) + \mathbb{E}[(\mathbf{X}^T \cdot \mathbf{X})^{-1} \cdot \mathbf{X}^T \cdot\vec{\epsilon}] \\ & as\ \vec{\beta} \ is \ constant\ and\ \mathbf{X}\ is\ also\ constant\\ & = \vec{\beta} + (\mathbf{X}^T \cdot \mathbf{X})^{-1} \cdot \mathbf{X}^T \cdot \mathbb{E}(\vec{\epsilon}) \\ \end{align*} $$

But some materials in econometrics stated the Assumption 2 or Assumption 3. I am confused as $\mathbf{X}$ is considered observed or fixed in OLS, conditioned expectation in Assumption 2 doesn't seem to make sense. And Assumption 3 should easily be simplified to Assumption 1 given $\mathbf{X}$ fixed.

$\endgroup$

1 Answer 1

3
$\begingroup$

If you regard $X$ as fixed then assumptions 1 and 2 are the same. Since $X$ often isn't fixed, just conditioned on, it's helpful to have the assumption in a form that doesn't rely on $X$ being fixed. That's the point of assumption 2. Both assumptions 1 and 2 are saying that $E[Y|X=x]$ is truly is $x\beta$.

Assumption 3 is importantly weaker. Assumptions 1 and 2 require each residual individually to have zero mean; assumption 3 only gives conditions on sums of residuals.

For example, suppose $E[Y|X=x]$ is actually a curve, so that $E[Y|X=x]\neq x\beta$. There will still be a best-fitting true line, a value of $\beta$ that makes $E[X^Te]=0$, but we will not have $E[e_i|X_i=x]=0$ for any individual point. In this case it is still true that $\hat\beta$ is unbiased for the slope of the true best-fitting line, but that line is not $E[Y|X=x]$.

$\endgroup$
4
  • $\begingroup$ Thx! This makes so much sense. But if we assume $\mathbf{X}$ to be random too: $$ \begin{align*} \mathbb{E}(\hat{\vec{\beta}}) = \vec{\beta} + \mathbb{E}_{X}[(\mathbf{X}^T \cdot \mathbf{X})^{-1} \cdot \mathbf{X}^T \cdot\mathbb{E}_{\vec{\epsilon}|X}(\vec{\epsilon}|\mathbf{X})] \end{align*} $$ When $\mathbb{E}_{\vec{\epsilon}|X}(\vec{\epsilon}|\mathbf{X}) = 0$, it is indeed unbiased (sufficient). But this isn't a necessary condition (non-zero inner expectations can cancel out in the outer one), right? $\endgroup$
    – Kay99
    Sep 12, 2023 at 19:34
  • $\begingroup$ No, neither assumption 1 nor assumption 2 is necessary: assumption 3 is weaker, and is necessary. $\endgroup$ Sep 12, 2023 at 22:04
  • $\begingroup$ Thanks a gain Thomas! I don't quite get how to derive assumption 3 when $\mathbf{X}$ is random. As $(\mathbf{X}^T \cdot \mathbf{X})^{-1}$ in $\mathbb{E}[(\mathbf{X}^T \cdot \mathbf{X})^{-1} \cdot \mathbf{X}^T \cdot\vec{\epsilon}]$ is also random, it seems we cannot take it out directly. How can we prove that $\mathbb{E}(\mathbf{X}^T \cdot \vec{\epsilon}) = \vec{0}$ will satisfy the unbiasness? $\endgroup$
    – Kay99
    Sep 13, 2023 at 0:51
  • $\begingroup$ Condition on $X$. It's unbiased for every fixed $X$ and therefore it's unbiased for random $X$. $\endgroup$ Sep 13, 2023 at 2:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.