# Minimum variance unbiased estimator

A. Determine the restrictions needed on the $a$'s so that $z=\sum_{i=1}^{n}a_iX_i$ will be an unbiased estimator $E\left[X\right]$, where $X_1,X_2,\dotsc,X_n$ represents a random sample of $X$.

Answer: The $a$'s must sum to one.

B. What is the best set of $a$'s to choose from A if $z=\sum_{i=1}^{n}a_iX_i$ is to be an unbiased estimator of $E\left[X\right]$ with minimum variance.

Answer: $a_i=1/n$.

The first answer I've shown I understand. However I'm a bit stumped as to how to arrive at the answer to part B.

EDIT: I'm not exactly sure what the question is asking. What I think it's asking is: find $$\left\{a_i:\mathrm{var}\left(\sum_{i=1}^{n}a_iX_i\right)=\frac{1}{n\sigma^2}\right\},$$ where $\sigma^2$ is the expected value of the squared derivative of some arbitrary log likelihood function of $X$ given some parameter $\theta$.

I know that $$\mathrm{var}\left(X\right)=E\left[\left(X-\theta\right)^2\right].$$ How do I compute the variance of $z$?

$\mathrm{var}\left(z\right) = E\left[z^2\right] - \theta^2$.

How do I evaluate $E\left[z^2\right]$ without a density function?

Thanks!

• What have you tried? Where are you stuck? Please provide some such context so that we can best help you. – cardinal Feb 19 '12 at 21:32

If the $X_i$ are iid each with positive finite variance $v$ then $$\text{var}\left(\sum_i a_i X_i\right) = \sum_i \text{var}\left( a_i X_i\right) = \sum_i a_i^2 \text{var}\left( X_i\right) = \sum_i a_i^2 v = v \sum_i a_i^2$$
so you want to minimise $v \sum_i a_i^2$ subject to $\sum_i a_i =1$ (since it has to be unbiased). You can ignore the positive constant $v$ and deduce this happens when each $a_i=1/n$; for example the Cauchy–Schwarz inequality will do this.
• Why can you pull out $v$? Is it because $v = n\cdot\mathrm{var}\left(X\right)$? – Phillip Cloud Feb 19 '12 at 23:41
• It is a multiplicative constant equal to $\text{var}\left( X_i\right)$ - I have added an extra step to make this clearer – Henry Feb 19 '12 at 23:47
• You are aiming to minimise $v \sum_i a_i^2$ subject to $\sum_i a_i =1$. The values of $a_i$ which do this also minimise $\sum_i a_i^2$ subject to $\sum_i a_i =1$ since $v$ is a positive constant. – Henry Feb 19 '12 at 23:54
• I'm still not getting it. If I write $2v\sum_{i=1}^{n}a_i=2$, since $\sum_{i=1}^{n}a_i=1$ that doesn't help me at all. – Phillip Cloud Feb 20 '12 at 21:15