# Gauss Markov Theorem for linear combinations [duplicate]

I know that the Gauss Markov theorem implies that under some conditions, OLS estimates have the smallest variance of all unbiased linear estimators. In particular if I have a model like $$y = \alpha + \beta_1.x_1 + \beta_2.x_2$$ then I know that my OLS estimate $$\hat{\beta}_1$$ has smallest variance and my estimate $$\hat{\beta}_2$$ has also smallest variance. But is it true that my estimate of $$\hat{\beta}_2 - \hat{\beta}_1$$ has the smallest variance?

• Use your algebra to observe $y = \alpha + (\beta_2-\beta_1)(x_2-x_1)/2+(\beta_2+\beta_1)(x_2+x_1)/2=\alpha+\alpha_1(x_2-x_1)/2+\alpha_2(x_2+x_1)/2$ and consider how your OLS estimates of the $\beta_i$ and $\alpha_i$ might be related.
– whuber
Feb 8 at 18:26
• Yes, this link gives a complete account on this. Feb 8 at 18:28
• @Zhanxiong this doesn't talk my question because in my case, $var(\beta_2 - \beta_1) = var(\beta_2) + var(\beta_1) - 2cov(\beta_2, \beta_1)$ and since the $cov$ is negative here, following the proofs, $cov(\beta_2, \beta_1)$ for OLS is the smallest, and so substracting that from the whole variance difference, leaves ambiguous effects no? Feb 8 at 18:58
• @jbowman thank you so much! But not really, since if you look at my comment just above this one, I have a negative sign in front of the covariance so this wouldn't really work, in fact leaves me with an ambiguous result. Feb 8 at 18:59
• Subtraction is the same as addition, just with a negative number. The point is that you have a linear combination of the parameters estimated, and that's what the proof covers - much more general than just subtracting one parameter from the other. Feb 8 at 19:40

$$\DeclareMathOperator{\Var}{Var}$$
Suppose that $$a'y$$ is an unbiased linear estimator of $$\beta_2 - \beta_1 =: c'\beta$$ where $$c = (0, -1, 1)'$$ and $$\beta = (\alpha, \beta_1, \beta_2)'$$. Note that, the following proof clearly generalizes to all linear functionals of $$\beta$$ and any design matrix $$X \in \mathbb{R}^{n \times p}$$ with $$\operatorname{rank}(X) = p$$.
By $$E[a'y] = c'\beta$$, it follows that $$a'X\beta = c'\beta$$, where $$X$$ is the design matrix. Because this holds for all $$\beta \in \mathbb{R}^p$$, this requires $$c = X'a$$. It then follows that \begin{align} & \Var(a'y) - \Var(c'\hat{\beta}) \\ =& \sigma^2(a'a - c'(X'X)^{-1}c) \\ =& \sigma^2(a'a - a'X(X'X)^{-1}X'a) \\ =& \sigma^2a'(I - H)a \\ =& \sigma^2 ((I - H)a)'((I - H)a) \geq 0. \end{align}
In the above derivation, $$H = X(X'X)^{-1}X'$$ is the hat matrix, which is idempotent and symmetric. Consequently, $$I - H$$ is idempotent, which entails the desired result.