İs this a valid algorithm for finding MVUE?

Question

Let $\theta$ be a parameter regarding a distribution, and $\hat \theta_1, \hat \theta_2$be two different unbiased estimators of $\theta$ with variance $\sigma_1, \sigma_2$. Then it is known that $$\hat \theta_3 = 0.5 \hat \theta_1 + 0.5 \hat \theta_2$$ is a new estimator with variance $\sigma_3 = \frac{\sigma_1}{\sigma_1 + \sigma_2}$.

However, now we can apply the same procedure for $\hat \theta_1, \hat \theta_3$ to get a new estimator with even smaller variance.

In this way, can we get MVUE of $\theta$ ?

Answer 1

Not necessarily. Here's a counterexample:

Let $X \sim \text{N}(\theta,1)$. Our two different unbiased estimators of $\theta$ are the average of the largest and smallest values of the sample (size $n$) $(\hat{\theta}_1 = (x_{[1]}+x_{[n]})/2)$ and the average of the second largest and second smallest values of the sample $(\hat{\theta}_2 = (x_{[2]}+x_{[n-1]})/2)$. Due to the symmetry of the Normal distribution, it should be clear that both estimators are unbiased. However, for all $n > 4$, we are throwing out all but the two smallest and two largest data points; it should be intuitively clear that there is no way to construct an estimator from these four data points that has the same variance as the MVUE, which is the sample mean $\bar{x} = \sum_{i=1}^nx_i / n$, and which takes into account all of the data points. (Consider the case $n=1,000,000$ if this isn't clear.)

We can also use the Rao-Blackwell theorem as a quick mathematical proof; I'll present a nonrigorous outline of such. Rao-Blackwell, in essence, states that any unbiased estimator can be improved by conditioning upon a sufficient statistic unless it is already a function of the sufficient statistic. In this case, the sufficient statistic is the sample mean; clearly, $\hat{\theta}_1$ and $\hat{\theta}_2$ are not functions of the sample mean, nor is any weighted average of the two. Therefore, Rao-Blackwell tells us, we can construct a better estimator than the weighted average by conditioning it upon $\bar{x}$, which in turn implies that no weighted average of $\hat{\theta}_1$ and $\hat{\theta}_2$ can be a MVUE.