Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Unbiased weighted variance was already addressed here and elsewhere but there still seems to be a surprising amount of confusion. There appears to be a consensus toward the formula presented in the first link as well as in the Wikipedia article. This also looks like the formula used by R, Mathematica, and GSL (but not MATLAB). However, the Wikipedia article also contains the following line which looks like a great sanity check for a weighted variance implementation:

For example, if values {2,2,4,5,5,5} are drawn from the same distribution, then we can treat this set as an unweighted sample, or we can treat it as the weighted sample {2,4,5} with corresponding weights {2,1,3}, and we should get the same results.

My calculations give the value of 2.1667 for variance of the original values and 2.9545 for the weighted variance. Should I really expect them to be the same? Why or why not?

share|improve this question
    
should probably me moved to SO. –  user603 Mar 6 '13 at 10:51
1  
this question is not really about implementation, but the theory behind it –  confusedCoder Mar 6 '13 at 16:31

1 Answer 1

Yes, you should expect both examples (unweighted vs weighted) to give you the same results.

I have implemented the two algorithms from the Wikipedia article.

This one works:

If all of the $x_i$ are drawn from the same distribution and the integer weights $w_i$ indicate frequency of occurrence in the sample, then the unbiased estimator of the weighted population variance is given by:

$s^2\ = \frac {1} {V_1 - 1} \sum_{i=1}^N w_i \left(x_i - \mu^*\right)^2,$

However this one (using fractional weights) does not work for me:

If each $x_i$ is drawn from a Gaussian distribution with variance $1/w_i$, the unbiased estimator of a weighted population variance is given by:

$s^2\ = \frac {V_1} {V_1^2-V_2} \sum_{i=1}^N w_i \left(x_i - \mu^*\right)^2$

I am still investigating the reasons why the second equation does not work as intended.

/EDIT: Found the reason why the second equation did not work as I thought: you can use the second equation only if you have normalized weights or variance ("reliability") weights, and it is NOT unbiased, because if you don't use "repeat" weights (counting the number of times an observation was observed and thus should be repeated in your math operations), you lose the ability to count the total number of observations, and thus you can't use a correction factor.

So this explains the difference in your results using weighted and non-weighted variance: your computation is biased.

Thus, if you want to have an unbiased weighted variance, use only "repeat" weights and use the first equation I have posted above. If that's not possible, well, you can't help it.

I have also updated the Wikipedia's article if you want more info: http://en.wikipedia.org/wiki/Weighted_arithmetic_mean#Weighted_sample_variance

And a linked article about unbiased weighted covariance (which in fact is the same variance due to Polarization Identity): Correct equation for weighted unbiased sample covariance

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.