1
$\begingroup$

Suppose I have n observations, which are all normally distributed with the same mean (which is unknown) but each has a different variance (the different variances could be called $v_1, ..., v_n$ for example, which are all assumed to be known).

What is the best estimate of the mean? And further, how does one compute the variance of the mean?

Clearly, the sample average would be the best estimate if iid, but intuitively, an observation which has a relatively low variance would provide more information about the mean than the others, suggesting that the sample mean would not be the best estimate. My problem is that I don't know how to 'quantify' this intuition to generate an estimate of the mean and it's variance. Any ideas on how to do this?

$\endgroup$
  • $\begingroup$ Maybe try different weights. What happens when you write down a general convex combination, then the "loss" of that? If you minimize your loss with respect to those weights and up with optimal weights that are proportional to the reciprocals of the variances, that would agree with your intuition I'll bet. $\endgroup$ – Taylor Sep 23 '17 at 14:41
4
$\begingroup$

You don't define how you want to measure best.

The easiest answer to give is a weighted average with the weights proportional to the inverse of the known variances. It will be best (minimum variance) among linear estimators (which are unbiased). This is best in the normal case by several common criteria.

$\hat{\mu} = \sum_i w_i x_i = \frac{1}{\sum_i 1/v_i}\sum_i x_i/v_i$

$\text{Var}(\hat{\mu})= (\frac{1}{\sum_i 1/v_i})^2 \sum_i 1/v_i^2 \cdot \text{Var}(x_i)$

$ = (\frac{1}{\sum_i 1/v_i})^2 \sum_i \frac{1}{v_i^2} \cdot v_i$

$ = (\frac{1}{\sum_i 1/v_i})^2 \sum_i 1/v_i$

$ = \frac{1}{\sum_i 1/v_i}$

$\endgroup$
  • 1
    $\begingroup$ +1 The observations are normal and independent so OP does have a distributional model. Hence the weighted average is the MLE and MVUE as well as the BLUE. Actually I can't immediately think of any sensible/usual criterion by which it wouldn't be the best estimator. $\endgroup$ – Gordon Smyth Sep 24 '17 at 4:23
  • 1
    $\begingroup$ Thanks @Gordon; I somehow missed the normal part. But one need only have a loss function like say the L1 norm to change the answer. $\endgroup$ – Glen_b Sep 24 '17 at 4:43
  • 1
    $\begingroup$ Sorry, don't agree. The weighted average is still the optimal estimator under the L1 loss function $E(|\hat\mu-\mu|)$. Think of it from a Bayesian point of view (with flat improper prior). The posterior distribution for $\mu$ is symmetric (normal in fact) with the weighted average $\hat\mu$ as both mean and median. Hence $\hat\mu$ must minimize both L1 and L2 loss functions. $\endgroup$ – Gordon Smyth Sep 24 '17 at 9:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.