Showing that the minimum-variance estimator is the OLS estimator

Question

Recap of required theory

Consider the following regression:

$$y_i = \alpha + \beta x_i + u_i \tag{1}$$

where $y_i$ are iid and $x_i$ are deterministic (i.e. fixed).

We know that the OLS estimator $\hat{\beta}$ is:

$$ \hat{\beta} = \frac{\sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^n (x_i - \bar{x})^2}$$

Assume that the Gauss-Markov conditions are fulfilled. Then, the OLS estimator has the minimum variance of any linear (in $y_i$) unbiased estimator.

Setup for my problem

Consider the following class of linear estimators:

$$ \tilde{\beta} = \frac{\sum_{i=1}^n a_i y_i}{\sum_{i=1}^n a_i x_i} \text{ where } \sum_{i=1}^n a_i = 0 \tag{2}$$

Notice that if we set $a_i = (x_i - \bar{x})$, we get the OLS estimator.

This class of estimators is unbiased; we can show this by substituting (1) into (2), leading to:

$$ \tilde{\beta} = \beta + \frac{\sum_{i=1}^n a_i u_i}{\sum_{i=1}^n a_i x_i} \tag{3}$$

Given that the error terms $u_i$ are exogenous, taking expectations through (3) completes the proof that this class of estimators is indeed unbiased.

Why have I bothered to show that this class of estimators is unbiased? Because if it is, then by the Gauss-Markov theorem:

Setting $a_i = (x_i - \bar{x})$ minimizes the variance of $\tilde{\beta}$.

This is the result I want to prove.

---

My problem: Minimizing the variance of $\tilde{\beta}$

Assuming homoskedasticity i.e $V(u_i) = \sigma^2$, we can use (3) to show that:

$$ V(\tilde{\beta}) = \frac{\sigma^2 \sum_{i=1}^n a_i^2}{\left[\sum_{i=1}^n a_i x_i\right]^2} \tag{4}$$

This is a one-variable optimization problem, since $x_i$ are fixed. Differentiating wrt $a_i$ using the quotient rule yields the first order condition:

$$ \frac{dV}{da} = \frac{2\sigma^2\left[\sum_{i=1}^n a_i x_i\right]^2\left[\sum_{i=1}^n a_i\right] - 2\sigma^2\left[\sum_{i=1}^n a_i x_i^2\right]\left[\sum_{i=1}^n a_i^2\right] }{\left[\sum_{i=1}^n a_i x_i\right]^4} = 0 \tag{5}$$

This simplifies down to:

$$ \left[\sum_{i=1}^n a_i x_i\right]^2\left[\sum_{i=1}^n a_i\right] = \left[\sum_{i=1}^n a_i x_i^2\right]\left[\sum_{i=1}^n a_i^2\right] \tag{6} $$

Using that $\sum_{i=1}^n a_i = 0$, this becomes:

$$\left[\sum_{i=1}^n a_i x_i^2\right]\left[\sum_{i=1}^n a_i^2\right] = 0 \tag{7}$$

...and this is where I get stuck.

How can I solve (7) to show that:

$$a_i = x_i - \bar{x}$$

I realize that this is simply a question of algebraic manipulation (I think I've derived everything up to (7) correctly); I just can't see how to simplify it.

Answer 1

Because $\sum_{i=1}^n a_i = 0$, $\sum_{i=1}^n a_i x_i =\sum_{i=1}^n a_i x_i -\bar x \sum_{i=1}^n a_i = \sum_{i=1}^n a_i (x_i-\bar x)$. So (4) can be written as $$V(\tilde{\beta}) = \frac{\sigma^2 \sum_{i=1}^n a_i^2}{\left[\sum_{i=1}^n a_i x_i\right]^2}=\frac{\sigma^2 \sum_{i=1}^n a_i^2}{\left[\sum_{i=1}^n a_i (x_i - \bar x)\right]^2}$$

According to Cauchy-Schwarz Inequality, we have $$\left[\sum_{i=1}^n a_i (x_i - \bar x)\right]^2 \le \sum_{i=1}^n (x_i-\bar x)^2\sum_{i=1}^n a_i^2$$ Divided by $\left[\sum_{i=1}^n a_i (x_i - \bar x)\right]^2 \sum_{i=1}^n (x_i-\bar x)^2$ on both sides, we get: $$\frac{\sum_{i=1}^n a_i^2}{\left[\sum_{i=1}^n a_i (x_i-\bar x)\right]^2} \ge \frac 1 {\sum_{i=1}^n (x_i-\bar x)^2} = \frac {\sum_{i=1}^n (x_i-\bar x)^2}{\left(\sum_{i=1}^n (x_i-\bar x)^2\right)^2}$$ So $V(\hat \beta) \le V(\tilde \beta)$