Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Say there are $m+n$ elements split into two groups ($m$ and $n$). The variance of first group is $\sigma_m^2$ and variance of second group is $\sigma^2_n$. The elements themselves are assumed to be unknown but I know the means $\mu_m$, $\mu_n$ and $\mu_{(m+n)}$. The variance doesn't have to be unbiased so denominator is $(m+n)$ and not $(m+n-1)$.

Is there a way to calculate the combined variance $\sigma^2_{(m+n)}$?

share|improve this question
    
When you say you know the means and variances of these groups, are they parameters or sample values? If they are sample means/variances you should not use $\mu$ and $\sigma$... –  Jonathan Christensen Nov 8 '12 at 19:18
    
I just used the symbols as a representation. Otherwise, it would have been hard to explain my problem. –  zepp Nov 9 '12 at 0:11
    
For sample values, we usually use Latin letters (e.g. $m$ and $s$). Greek letters are usually reserved for parameters. Using the "correct" (expected) symbols will help you communicate more clearly. –  Jonathan Christensen Nov 9 '12 at 2:31
    
No worries, I'll follow that from now on! Cheers –  zepp Nov 9 '12 at 8:42
    
@Jonathan Because this is not a question about samples or estimation, one can legitimately take the view that $\mu$ and $\sigma^2$ are the true mean and variance of the empirical distribution of a batch of data, thereby justifying the conventional use of greek letters rather than latin letters to refer to them. –  whuber Nov 9 '12 at 18:36
show 3 more comments

2 Answers

up vote 6 down vote accepted

Use the definitions of sample variance and mean to find the sum of squares of all the data $x_i$ (where $i=1,\ldots,n$ designates elements of the first group and $i=n+1,\ldots,n+m$ designates elements of the second group), then break that sum by group and re-express the two pieces in terms of the variances and means of the subsets of the data:

$$\eqalign{ (m+n)(\sigma^2_{m+n} + \mu_{m+n}^2) &= \sum_{i=1}^{n+m} x_i^2 \\ &= \sum_{i=1}^n x_i^2 + \sum_{i=n+1}^{n+m} x_i^2 \\ &= n(\sigma^2_n + \mu_n^2) + m(\sigma^2_m + \mu_m^2). }$$

Algebraically solving this for $\sigma^2_{m+n}$ in terms of the other (known) quantities yields

$$\sigma^2_{m+n} = \frac{n(\sigma^2_n + \mu_n^2) + m(\sigma^2_m + \mu_m^2)}{m+n} - \mu^2_{m+n}.$$

share|improve this answer
    
Hi, thanks for your reply. I didn't tag this question as homework. It kinda feels stupid when it is tagged as homework when I am actually doing a PhD and was stumped upon trying to find a solution to this question to use it in one of my experiments. However, that aside, thanks again for your help. It definitely cleared things up for me. –  zepp Nov 9 '12 at 0:10
2  
The "homework" tag doesn't mean the question is elementary or stupid: it's used for self-study questions that can even include research-level queries. It distinguishes routine, more or less context-free questions (of the sort that might ordinarily grace the math forum) from specific applied questions. –  whuber Nov 9 '12 at 18:33
add comment

Yes, given the mean, sample count, and variance or standard deviation of each of two or more groups of samples, you can exactly calculate the variance or standard deviation of the combined group.

This web page describes how to do it, and why it works; it also includes source code in Perl: http://www.burtonsys.com/climate/composite_standard_deviations.html


BTW, contrary to the answer given above,

$$\eqalign{ n(\sigma^2 + \mu^2) \space\space \ne \space\space \sum_{i=1}^n x_i^2 }$$

See for yourself, e.g., in R:

> x = rnorm(10,5,2)
> x
 [1] 6.515139 8.273285 2.879483 3.624233 6.199610 3.683164 4.921028 8.084591
 [9] 2.974520 6.049962
> mean(x)
[1] 5.320502
> sd(x)
[1] 2.007519
> sum(x**2)
[1] 319.3486
> 10 * (mean(x)**2 + sd(x)**2)
[1] 323.3787
share|improve this answer
    
it's because you forgot the n-1 factor, e.g. try with n*(mean(x)**2+sd(x)**2/(n)*(n-1)) –  user603 May 13 '13 at 20:11
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.