I know that the Gauss Markov theorem implies that under some conditions, OLS estimates have the smallest variance of all unbiased linear estimators. In particular if I have a model like $ y = \alpha + \beta_1.x_1 + \beta_2.x_2$ then I know that my OLS estimate $\hat{\beta}_1$ has smallest variance and my estimate $\hat{\beta}_2$ has also smallest variance. But is it true that my estimate of $\hat{\beta}_2 - \hat{\beta}_1$ has the smallest variance?
10
-
1$\begingroup$ Use your algebra to observe $y = \alpha + (\beta_2-\beta_1)(x_2-x_1)/2+(\beta_2+\beta_1)(x_2+x_1)/2=\alpha+\alpha_1(x_2-x_1)/2+\alpha_2(x_2+x_1)/2$ and consider how your OLS estimates of the $\beta_i$ and $\alpha_i$ might be related. $\endgroup$– whuber ♦Feb 8 at 18:26
-
1$\begingroup$ Yes, this link gives a complete account on this. $\endgroup$– ZhanxiongFeb 8 at 18:28
-
$\begingroup$ @Zhanxiong this doesn't talk my question because in my case, $var(\beta_2 - \beta_1) = var(\beta_2) + var(\beta_1) - 2cov(\beta_2, \beta_1)$ and since the $cov$ is negative here, following the proofs, $cov(\beta_2, \beta_1)$ for OLS is the smallest, and so substracting that from the whole variance difference, leaves ambiguous effects no? $\endgroup$– Lola1993Feb 8 at 18:58
-
$\begingroup$ @jbowman thank you so much! But not really, since if you look at my comment just above this one, I have a negative sign in front of the covariance so this wouldn't really work, in fact leaves me with an ambiguous result. $\endgroup$– Lola1993Feb 8 at 18:59
-
1$\begingroup$ Subtraction is the same as addition, just with a negative number. The point is that you have a linear combination of the parameters estimated, and that's what the proof covers - much more general than just subtracting one parameter from the other. $\endgroup$– jbowmanFeb 8 at 19:40
|
Show 5 more comments
1 Answer
$\begingroup$
$\endgroup$
$\DeclareMathOperator{\Var}{Var}$
Suppose that $a'y$ is an unbiased linear estimator of $\beta_2 - \beta_1 =: c'\beta$ where $c = (0, -1, 1)'$ and $\beta = (\alpha, \beta_1, \beta_2)'$. Note that, the following proof clearly generalizes to all linear functionals of $\beta$ and any design matrix $X \in \mathbb{R}^{n \times p}$ with $\operatorname{rank}(X) = p$.
By $E[a'y] = c'\beta$, it follows that $a'X\beta = c'\beta$, where $X$ is the design matrix. Because this holds for all $\beta \in \mathbb{R}^p$, this requires $c = X'a$. It then follows that \begin{align} & \Var(a'y) - \Var(c'\hat{\beta}) \\ =& \sigma^2(a'a - c'(X'X)^{-1}c) \\ =& \sigma^2(a'a - a'X(X'X)^{-1}X'a) \\ =& \sigma^2a'(I - H)a \\ =& \sigma^2 ((I - H)a)'((I - H)a) \geq 0. \end{align}
In the above derivation, $H = X(X'X)^{-1}X'$ is the hat matrix, which is idempotent and symmetric. Consequently, $I - H$ is idempotent, which entails the desired result.
