13
$\begingroup$

I have a large set of data points of the form (mean, stdev). I wish to reduce this to a single (better) mean, and a (hopefully) smaller standard deviation.

Clearly I could simply compute $\frac{\sum data_{mean}}{N}$, however this does not take in to account the fact that some of the data points are significantly more accurate than others.

To put it simply, I wish to preform a weighted average of these data points, but do not know what the weighting function should be in terms of the standard deviation.

$\endgroup$
24
$\begingroup$

You seek a linear estimator for the mean $\mu$ of the form

$$\hat{\mu} = \sum_{i=1}^{n} \alpha_i x_i$$

where the $\alpha_i$ are the weights and the $x_i$ are the observations. The objective is to find appropriate values for the weights. Let $\sigma_i$ be the true standard deviation of $x_i$, which might or might not coincide with the estimated standard deviation you likely have. Assume the observations are unbiased; that is, their expectations all equal the mean $\mu$. In these terms we can compute that the expectation of $\hat{\mu}$ is

$$\mathbb{E}[\hat{\mu}] = \sum_{i=1}^{n} \alpha_i \mathbb{E}[x_i] = \mu \sum_{i=1}^{n} \alpha_i$$

and (provided the $x_i$ are uncorrelated) the variance of this estimator is

$$\operatorname{Var}\left[\hat{\mu}\right] = \sum_{i=1}^{n} \alpha_i^2 \sigma_i^2.$$

At this point many people require that the estimator be unbiased; that is, we want its expectation to equal the true mean. This implies the weights must sum to unity. Subject to this restriction, the accuracy of the estimator (as measured with mean square error) is optimized by minimizing the variance. The unique solution (easily obtained with a Lagrange multiplier or by re-interpreting the situation geometrically as a distance minimization problem) is that the weights $\alpha_i$ must be proportional to $1/\sigma_i^2$. The sum-to-unity restriction pins down their values, yielding

$$\hat{\mu} = \frac{\sum_{i=1}^{n} x_i / \sigma_i^2}{\sum_{i=1}^{n} 1 / \sigma_i^2}$$

and

$$\operatorname{Var}\left[\hat\mu\right] = \frac{1}{\sum_{i=1}^{n} 1 / \sigma_i^2} = \frac{1}{n} \left(\frac{1}{n} \sum_{i=1}^n \frac{1}{\sigma_i^2}\right)^{-1}.$$

In words,

the minimum-variance unbiased estimator of the mean is obtained by making the weights inversely proportional to the variances; the variance of that estimator is $1/n$ times the harmonic mean of the variances.

We usually do not know the true variances $\sigma_i$. About all we can do is to make the weights inversely proportional to the estimated variances (the squares of your standard deviations) and trust this will work well.

$\endgroup$
  • 1
    $\begingroup$ and related to this answer, also from whuber: stats.stackexchange.com/questions/9071/… $\endgroup$ – Henry Sep 6 '11 at 21:46
  • $\begingroup$ What would happen if we do not "Assume the observations are unbiased"? With that statement you are saying that if infinite random individual measurements are add to the observation $x_i$ we get the mean mu ? $\endgroup$ – user1420303 Nov 9 '15 at 1:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.