0
$\begingroup$

I saw two different derivations of $E[\hat{\beta}] = \beta$, and they don't appear to be equivalent to me.

Method 1 (from https://www.youtube.com/watch?v=T5kjKqkCvHc)

Method 2 (from https://www.statlect.com/fundamentals-of-statistics/Gauss-Markov-theorem#hid2, under "OLS is linear and unbiased", click on the "Proof")

The latter method first computes $E[\hat{\beta}|X] = \beta$, and then use the law of iterated expectations to obtain $E[\hat{\beta}] = \beta$. The former directly computes this without conditioning on $X$. Are these 2 different ways to arrive at the same answer, or is one of the methods arriving at the right answer using incorrect logic?

Correct me if I'm wrong, but it looks like the latter uses $E[\epsilon |X] = 0$ whereas the former uses $E[\epsilon] = 0$. Why do we need to condition on $X$ in the former case? That doesn't seem to be an assumption made by the gauss-markov theorem.

Improve this question
$\endgroup$
4
  • $\begingroup$ If you assume independence of $X$ and $\epsilon$, then $E(\epsilon)=E(\epsilon|X)$. In most of the literature though, $X$ is actually assumed non-random, and hence it doesn't matter whether you specifically write down the dependence on $X$. $\endgroup$
    – Tim Mak
    May 18, 2020 at 7:45
  • $\begingroup$ @TimMak Hmm, I see, but when I read up on the Gauss-Markov theorem, I didn't see an assumption of independence between $X$ and $\epsilon$. Only between $\epsilon_i, \epsilon_j$ for $i \neq j$. But I do recall seeing a source saying something about assuming $X$ is fixed. When you assume a random variable is fixed, does that affectively make it independent from other random variables? $\endgroup$
    – roulette01
    May 18, 2020 at 15:01
  • $\begingroup$ @TimMak I listened to the youtube video a bit more closely. Starting at 2:20, he states that $\epsilon$ (he uses the symbol $u$ is independent of the $X$ terms, and that apparently this allows us to write $E[(X^TX)^{-1}X^T\epsilon] = (X^TX)^{-1}X^T E[\epsilon] $. I don't think this is sufficient? If they are independent, then we can write $E[(X^TX)^{-1}X^T\epsilon] = E[(X^TX)^{-1}X^T] E[\epsilon] $. The expectation can't be dropped on the $X$ terms until we know they're constant/fixed/not random. $\endgroup$
    – roulette01
    May 18, 2020 at 15:30
  • $\begingroup$ You are right. If $X$ and $\epsilon$ are independent, you ought to write $E((X^TX)^{-1}X^T)E(\epsilon)$. If $X$ is fixed, you write $(X^TX)^{-1}X^TE(\epsilon)$. In standard Gauss Markov, $X$ is as asumed fixed. The $X$ random case is beyond Gauss Markov. $\endgroup$
    – Tim Mak
    May 19, 2020 at 3:10

1 Answer 1

Reset to default
1
$\begingroup$

You can avoid conditioning on X in the proof only if you are assuming that X is fixed (i.e., non-random, a constant) or that X and the error term epsilon are independent. You need these assumptions to bring X outside the expectation operator. But these are very restrictive assumptions that can be replaced by the weaker assumption that the expected value of the error term epsilon given X is equal to zero.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks, is it a formal assumption that $X$ is fixed for the Gauss-Markov theorem? This seems to be implied by Tim Makeshifts, but various resources seem to not provide definitive information on this. $\endgroup$
    – roulette01
    May 19, 2020 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.