# Proof that least squares estimators are unbiased under gauss-markov assumptions

I saw two different derivations of $$E[\hat{\beta}] = \beta$$, and they don't appear to be equivalent to me.

# Method 2 (from https://www.statlect.com/fundamentals-of-statistics/Gauss-Markov-theorem#hid2, under "OLS is linear and unbiased", click on the "Proof")

The latter method first computes $$E[\hat{\beta}|X] = \beta$$, and then use the law of iterated expectations to obtain $$E[\hat{\beta}] = \beta$$. The former directly computes this without conditioning on $$X$$. Are these 2 different ways to arrive at the same answer, or is one of the methods arriving at the right answer using incorrect logic?

Correct me if I'm wrong, but it looks like the latter uses $$E[\epsilon |X] = 0$$ whereas the former uses $$E[\epsilon] = 0$$. Why do we need to condition on $$X$$ in the former case? That doesn't seem to be an assumption made by the gauss-markov theorem.

• If you assume independence of $X$ and $\epsilon$, then $E(\epsilon)=E(\epsilon|X)$. In most of the literature though, $X$ is actually assumed non-random, and hence it doesn't matter whether you specifically write down the dependence on $X$. Commented May 18, 2020 at 7:45
• @TimMak Hmm, I see, but when I read up on the Gauss-Markov theorem, I didn't see an assumption of independence between $X$ and $\epsilon$. Only between $\epsilon_i, \epsilon_j$ for $i \neq j$. But I do recall seeing a source saying something about assuming $X$ is fixed. When you assume a random variable is fixed, does that affectively make it independent from other random variables? Commented May 18, 2020 at 15:01
• @TimMak I listened to the youtube video a bit more closely. Starting at 2:20, he states that $\epsilon$ (he uses the symbol $u$ is independent of the $X$ terms, and that apparently this allows us to write $E[(X^TX)^{-1}X^T\epsilon] = (X^TX)^{-1}X^T E[\epsilon]$. I don't think this is sufficient? If they are independent, then we can write $E[(X^TX)^{-1}X^T\epsilon] = E[(X^TX)^{-1}X^T] E[\epsilon]$. The expectation can't be dropped on the $X$ terms until we know they're constant/fixed/not random. Commented May 18, 2020 at 15:30
• You are right. If $X$ and $\epsilon$ are independent, you ought to write $E((X^TX)^{-1}X^T)E(\epsilon)$. If $X$ is fixed, you write $(X^TX)^{-1}X^TE(\epsilon)$. In standard Gauss Markov, $X$ is as asumed fixed. The $X$ random case is beyond Gauss Markov. Commented May 19, 2020 at 3:10

• Thanks, is it a formal assumption that $X$ is fixed for the Gauss-Markov theorem? This seems to be implied by Tim Makeshifts, but various resources seem to not provide definitive information on this. Commented May 19, 2020 at 15:06