2
$\begingroup$

A. Determine the restrictions needed on the $a$'s so that $z=\sum_{i=1}^{n}a_iX_i$ will be an unbiased estimator $E\left[X\right]$, where $X_1,X_2,\dotsc,X_n$ represents a random sample of $X$.

Answer: The $a$'s must sum to one.

B. What is the best set of $a$'s to choose from A if $z=\sum_{i=1}^{n}a_iX_i$ is to be an unbiased estimator of $E\left[X\right]$ with minimum variance.

Answer: $a_i=1/n$.

The first answer I've shown I understand. However I'm a bit stumped as to how to arrive at the answer to part B.

EDIT: I'm not exactly sure what the question is asking. What I think it's asking is: find $$\left\{a_i:\mathrm{var}\left(\sum_{i=1}^{n}a_iX_i\right)=\frac{1}{n\sigma^2}\right\},$$ where $\sigma^2$ is the expected value of the squared derivative of some arbitrary log likelihood function of $X$ given some parameter $\theta$.

I know that $$\mathrm{var}\left(X\right)=E\left[\left(X-\theta\right)^2\right].$$ How do I compute the variance of $z$?

$\mathrm{var}\left(z\right) = E\left[z^2\right] - \theta^2$.

How do I evaluate $E\left[z^2\right]$ without a density function?

Thanks!

$\endgroup$
1
  • $\begingroup$ What have you tried? Where are you stuck? Please provide some such context so that we can best help you. $\endgroup$
    – cardinal
    Commented Feb 19, 2012 at 21:32

1 Answer 1

2
$\begingroup$

If the $X_i$ are iid each with positive finite variance $v$ then $$\text{var}\left(\sum_i a_i X_i\right) = \sum_i \text{var}\left( a_i X_i\right) = \sum_i a_i^2 \text{var}\left( X_i\right) = \sum_i a_i^2 v = v \sum_i a_i^2$$

so you want to minimise $v \sum_i a_i^2$ subject to $\sum_i a_i =1$ (since it has to be unbiased). You can ignore the positive constant $v$ and deduce this happens when each $a_i=1/n$; for example the Cauchy–Schwarz inequality will do this.

$\endgroup$
9
  • $\begingroup$ Why can you pull out $v$? Is it because $v = n\cdot\mathrm{var}\left(X\right)$? $\endgroup$ Commented Feb 19, 2012 at 23:41
  • $\begingroup$ It is a multiplicative constant equal to $\text{var}\left( X_i\right)$ - I have added an extra step to make this clearer $\endgroup$
    – Henry
    Commented Feb 19, 2012 at 23:47
  • $\begingroup$ Alright maybe I'm being thickheaded here, but what variable am I supposed to minimize with respect to if I'm ignoring the variance? $\endgroup$ Commented Feb 19, 2012 at 23:52
  • $\begingroup$ You are aiming to minimise $v \sum_i a_i^2$ subject to $\sum_i a_i =1$. The values of $a_i$ which do this also minimise $\sum_i a_i^2$ subject to $\sum_i a_i =1$ since $v$ is a positive constant. $\endgroup$
    – Henry
    Commented Feb 19, 2012 at 23:54
  • $\begingroup$ I'm still not getting it. If I write $2v\sum_{i=1}^{n}a_i=2$, since $\sum_{i=1}^{n}a_i=1$ that doesn't help me at all. $\endgroup$ Commented Feb 20, 2012 at 21:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.