# Determining true mean from noisy observations

I have a large set of data points of the form (mean, stdev). I wish to reduce this to a single (better) mean, and a (hopefully) smaller standard deviation.

Clearly I could simply compute $\frac{\sum data_{mean}}{N}$, however this does not take in to account the fact that some of the data points are significantly more accurate than others.

To put it simply, I wish to preform a weighted average of these data points, but do not know what the weighting function should be in terms of the standard deviation.

-

You seek a linear estimator for the mean $\mu$ of the form

$$\hat{\mu} = \sum_{i=1}^{n} \alpha_i x_i$$

where the $\alpha_i$ are the weights and the $x_i$ are the observations. The objective is to find appropriate values for the weights. Let $\sigma_i$ be the true standard deviation of $x_i$, which might or might not coincide with the estimated standard deviation you likely have. Assume the observations are unbiased; that is, their expectations all equal the mean $\mu$. In these terms we can compute that the expectation of $\hat{\mu}$ is

$$\mathbb{E}[\hat{\mu}] = \sum_{i=1}^{n} \alpha_i \mathbb{E}[x_i] = \mu \sum_{i=1}^{n} \alpha_i$$

and (provided the $x_i$ are uncorrelated) the variance of this estimator is

$$Var[\hat{\mu}] = \sum_{i=1}^{n} \alpha_i^2 \sigma_i^2 .$$

At this point many people require that the estimator be unbiased; that is, we want its expectation to equal the true mean. This implies the weights must sum to unity. Subject to this restriction, the accuracy of the estimator (as measured with mean square error) is optimized by minimizing the variance. The unique solution (easily obtained with a Lagrange multiplier or by re-interpreting the situation geometrically as a distance minimization problem) is that the weights $\alpha_i$ must be proportional to $1/\sigma_i^2$. The sum-to-unity restriction pins down their values, yielding

$$\hat{\mu} = \frac{\sum_{i=1}^{n} x_i / \sigma_i^2}{\sum_{i=1}^{n} 1 / \sigma_i^2}.$$

In other words, the minimum-variance unbiased estimator of the mean is obtained by making the weights inversely proportional to the variances.

We usually do not know the true variances $\sigma_i$. About all we can do is to make the weights inversely proportional to the estimated variances (the squares of your standard deviations) and trust this will work well.

-
+1, That is the most clear explanation I've ever seen. – Max Jun 23 '11 at 3:06
and related to this answer, also from whuber: stats.stackexchange.com/questions/9071/… – Henry Sep 6 '11 at 21:46