1
$\begingroup$

Total sample = subsample 1 + subsample 2

I have the mean and variance of the total sample and subsample 1. I have n for total, subsample 1, and subsample 2.

I can calculate the mean of subsample 2 thusly: (total mean * n total - subsample mean * n subsample 1)/n subsample 2

How can I calculate the variance of subsample 2?

$\endgroup$

1 Answer 1

1
$\begingroup$

Let me just introduce some notation first to make this easier. Let's denote the total number of data points in the full sample as $N$. The variance of the full sample is

$$\sigma_N^2 = \frac{1}{N} \sum_{n=0}^{N} (x_i - \mu_N)^2$$

for the total sample mean $\mu_N$, where you might change the normalisation factor to $N-1$ for an unbiased estimator. Similarly, we will denote the subsample variances and means as $\sigma_A^2$, $\sigma_B^2$ and $\mu_A$, $\mu_B$ and we'll say that the number of draws in subsample $A$ is $n_A$, leaving subsample $B$ with $n_B = N - n_A$.

If we expand the above expression, we get

$$\sigma_N^2 = \frac{1}{N} \sum_{n=0}^N(x_i^2 - 2\mu_Nx_i + \mu_N^2)$$

and bringing the sum through gives us

$$\sigma_N^2 = \frac{1}{N} \sum_{n=0}^{N} x_i^2 - 2\mu_N \left( \frac{1}{N} \sum_{n=0}^{N} x_i \right) +\frac{1}{N}\sum_{n=0}^{N}\mu_N^2 $$

If we add $\mu_N^2$ to both sides we are left with

$$\sigma_N^2 + \mu_N^2 = \frac{1}{N} \sum_{n=0}^N x_i^2$$

which we can do similarly with the subsamples. Splitting the sum in two over the subsamples, we now get

$$\sigma_N^2 + \mu_N^2 = \frac{1}{N} \left[\sum_{n=0}^{n_A} x_i^2 + \sum_{n=n_m + 1}^{N}x_i^2 \right]$$

and we can subsititute the sums for the identity derived above in each subsample

$$\sigma_N^2 + \mu_N^2 = \frac{1}{N} \left[ n_A \left( \sigma_A^2 + \mu_A^2 \right) + n_B \left( \sigma_B^2 + \mu_B^2 \right) \right] $$

So finally, after rearranging, we can get the variance of the second subsample as a function of the means and other variances:

$$\sigma_B^2 = \frac{N\left(\sigma_N^2 + \mu_N^2\right) - n_A \left(\sigma_A^2 + \mu_A^2 \right)}{n_B} - \mu_B^2$$

This will be slightly different if you are using unbiased estimators for the variance. You can see extra info relating to this problem in answers to the following other questions:

How to calculate the variance of a partition of variables

How to calculate pooled variance of two groups given known group variances, means, and sample sizes?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.