A. Determine the restrictions needed on the $a$'s so that $z=\sum_{i=1}^{n}a_iX_i$ will be an unbiased estimator $E\left[X\right]$, where $X_1,X_2,\dotsc,X_n$ represents a random sample of $X$.

Answer: The $a$'s must sum to one.

B. What is the best set of $a$'s to choose from A if $z=\sum_{i=1}^{n}a_iX_i$ is to be an unbiased estimator of $E\left[X\right]$ with minimum variance.

Answer: $a_i=1/n$.

The first answer I've shown I understand. However I'm a bit stumped as to how to arrive at the answer to part B.

EDIT: I'm not exactly sure what the question is asking. What I think it's asking is: find $$\left\{a_i:\mathrm{var}\left(\sum_{i=1}^{n}a_iX_i\right)=\frac{1}{n\sigma^2}\right\},$$ where $\sigma^2$ is the expected value of the squared derivative of some arbitrary log likelihood function of $X$ given some parameter $\theta$.

I know that $$\mathrm{var}\left(X\right)=E\left[\left(X-\theta\right)^2\right].$$ How do I compute the variance of $z$?

$\mathrm{var}\left(z\right) = E\left[z^2\right] - \theta^2$.

How do I evaluate $E\left[z^2\right]$ without a density function?


  • $\begingroup$ What have you tried? Where are you stuck? Please provide some such context so that we can best help you. $\endgroup$ – cardinal Feb 19 '12 at 21:32

If the $X_i$ are iid each with positive finite variance $v$ then $$\text{var}\left(\sum_i a_i X_i\right) = \sum_i \text{var}\left( a_i X_i\right) = \sum_i a_i^2 \text{var}\left( X_i\right) = \sum_i a_i^2 v = v \sum_i a_i^2$$

so you want to minimise $v \sum_i a_i^2$ subject to $\sum_i a_i =1$ (since it has to be unbiased). You can ignore the positive constant $v$ and deduce this happens when each $a_i=1/n$; for example the Cauchy–Schwarz inequality will do this.

| cite | improve this answer | |
  • $\begingroup$ Why can you pull out $v$? Is it because $v = n\cdot\mathrm{var}\left(X\right)$? $\endgroup$ – Phillip Cloud Feb 19 '12 at 23:41
  • $\begingroup$ It is a multiplicative constant equal to $\text{var}\left( X_i\right)$ - I have added an extra step to make this clearer $\endgroup$ – Henry Feb 19 '12 at 23:47
  • $\begingroup$ Alright maybe I'm being thickheaded here, but what variable am I supposed to minimize with respect to if I'm ignoring the variance? $\endgroup$ – Phillip Cloud Feb 19 '12 at 23:52
  • $\begingroup$ You are aiming to minimise $v \sum_i a_i^2$ subject to $\sum_i a_i =1$. The values of $a_i$ which do this also minimise $\sum_i a_i^2$ subject to $\sum_i a_i =1$ since $v$ is a positive constant. $\endgroup$ – Henry Feb 19 '12 at 23:54
  • $\begingroup$ I'm still not getting it. If I write $2v\sum_{i=1}^{n}a_i=2$, since $\sum_{i=1}^{n}a_i=1$ that doesn't help me at all. $\endgroup$ – Phillip Cloud Feb 20 '12 at 21:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.