1
$\begingroup$

So I read online that one of the assumptions of Gauss Markov Theorem is: $$E[\epsilon_i]=0$$However, we also know that one of the assumptions for linear regression is the zero conditional mean: $$E[\epsilon|X]=0$$ So I'm wondering, why the difference here? Do we have to further assume mean independence for the first case, i.e. $$E[\epsilon|X]=E[\epsilon ]$$ to arrive at the strict exogeneity criteria? Also, whats the benefit of making these 2 assumptions, as opposed to just assuming $E[\epsilon|X]=0$?

A related discussion is here: What's the difference between "mean independent" and independent? where carlos says "mean independence is not an assumption for linear regression" which further confuses me.

$\endgroup$

1 Answer 1

1
$\begingroup$

Comment:

The concept of mean independence is often used in statistics with two implications:

  • strong assumption of independent random variables $\left(X_{1} \perp X_{2}\right)$
  • weak assumption of uncorrelated random variables $\left(\operatorname{Cov}\left(X_{1}, X_{2}\right)=0\right)$.

There's nothing outside of these two implications.

So what is?

$\mathbb E[X_1 X_2]=\mathbb E[X_1] \mathbb E[X_2]$

This is uncorrelated random variables.

How we know this?

$\operatorname{Cov}(X_1 X_2)=\mathbb E[X_1 X_2]-\mathbb E[X_1] \mathbb E[X_2]=0$

So know we know two RVs are not correlated what is independent?

Let's say first: RVs that are independent are by definition uncorrelated. So being independent assumes uncorrelated. But what is independent?

Two RVs $X_1$ and $X_2$ are independent if their joint distribution is equal to the product of their marginal distributions:

$\mathbb P(x_1,x_2) = \mathbb P(x_1)\mathbb P(x_2)$ , where $x_1 \sim X_1$ and $x_2 \sim X_2$

In other words, having information on $X_1$ provides no additional information on $X_2$ and vice versa:

$\mathbb P(X_1 \mid X_2) = \mathbb P(X_1)$

$\mathbb P(X_2 \mid X_1) = \mathbb P(X_2)$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.