2
$\begingroup$

Could someone explain for me what the difference between Combined variance and pooled variance is? I have couple of groups (more than 2) with different sample size, I want to calculate the overall variance, Std, SE and CI... May I know which method is more appropriate for overall variance?

I found below formula:

$$S_c^2 = \frac{n_1[S_1^2+(\bar X_1 - \bar X_C)^2] +n_2[S_2^2+(\bar X_2 - \bar X_C)^2] }{n_1 + n_2}\,,$$

where $$\bar X_c=\frac{n_1\bar X_1+n_2\bar X_2}{n_1+n_2}$$

$$ S_p^2 = \frac{(n-1)S_x^2 + (m-1)S_y^2}{n + m -2}$$

$\endgroup$
5
  • $\begingroup$ Here is a MathJax tutorial for typesetting math: math.meta.stackexchange.com/q/5020/321264. $\endgroup$ Commented Dec 17, 2021 at 14:24
  • $\begingroup$ Thank you. Should I edit my post? $\endgroup$
    – Ati
    Commented Dec 17, 2021 at 14:39
  • $\begingroup$ Editing has been done. $\endgroup$ Commented Dec 17, 2021 at 14:52
  • $\begingroup$ There isn't any difference between "pooled" and "combined". Your $S_c^2$ is a valid expression in general when you just have some data. The $S_p^2$ is usually used in the context of inference on the unknown population variance. Both are for two groups of observations. $\endgroup$ Commented Dec 17, 2021 at 15:10
  • $\begingroup$ Thank you for your reply. I thought the same that combined and pooled variance are the same, but when I applied these two formula on my samples, got completely different numbers. $\endgroup$
    – Ati
    Commented Dec 17, 2021 at 15:43

1 Answer 1

3
$\begingroup$

The above commenter's statement is incorrect. There are two differences between the formula you are calling "combined" variance and the formula you are calling "pooled" variance. One difference is relatively minor, while another difference is more substantive. To homogenize notation, let me index my groups as 1 and 2 in the pooled variance case as well, so $$S_p^2 = \frac{(n_1-1)S_1^2 + (n_2 - 1) S_2^2}{n_1 + n_2 - 2}$$ A first difference between the pooled and combined variance formulas you have displayed is that the pooled variance formula given here contains what is known as a "degrees of freedom" correction. The point is that the usual sample variance formula divides by $n-1$ instead of $n$, as doing so produces an unbiased estimate for the population variance. When $n$ is large, this distinction hardly matters, so in discussing the second distinction, I will largely ignore it and consider the following version of the pooled variance formula $$S_p^2 = \frac{n_1 S_1^2 + n_2 S_2^2}{n_1 + n_2}$$ After making this change, it is pretty easy to see that this pooled variance formula is still not the same as the combined variance formula. To understand the distinction, it is useful to think about populations rather than samples. In population, a well known result in probability is the so-called law of total variance, which states that for random variables $Y,X$, $$\mathrm{Var}(X) = \mathbb E[\mathrm{Var}(X|Y)] + \mathrm{Var}(\mathbb E[X | Y])$$ In the context of pooled/combined variances, we can take $X$ to be the outcome which you are measuring and $Y$ to be which group somebody belongs to, so $Y = 1$ or $Y= 2$. The law of total variance would then read $$\begin{aligned}\mathrm{Var}(X) &= \mathrm{Var}(X | Y = 1) \mathrm{Pr}(Y = 1) + \mathrm{Var}(X | Y = 2) \mathrm{Pr}(Y = 2)\\ &= \left(\mathbb E[X | Y = 1] - \mathbb E[X]\right)^2\mathrm{Pr}(Y = 1) + \left(\mathbb E[X | Y = 2] - \mathbb E[X]\right)^2\mathrm{Pr}(Y = 2)\end{aligned}$$ Now, with this formula in mind, we can better understand the formula for $S_c^2$. Note the following:

  1. $n_1 / (n_1 + n_2)$ is the sample analogue of $\mathrm{Pr}(Y = 1)$ while $n_2 / (n_1 + n_2)$ is the sample analogue of $\mathrm{Pr}(Y = 1)$
  2. $S_1^2$ is the sample analogue for $\mathrm{Var}(X | Y = 1)$ while $S_2^2$ is the sample analogue for $\mathrm{Var}(X | Y = 2)$.
  3. $\bar X_1$ is the sample analogue of $\mathbb E[X | Y = 1]$ while $\bar X_2$ is the sample analogue of $\mathbb E[X | Y = 2]$.
  4. $\bar X_c$ is the sample analogue of $\mathbb E[X]$.

From this, you can see that $S_c^2$ simply decomposes the variance of the outcome $X$ in terms of its "within group" and "between group" components. By contrast, $S_p^2$ only includes the variance within group. Thus, $S_c^2$ and $S_p^2$ correspond to subtly different questions. In particular, $S_c^2$ asks "if I drew a random observation from the entire population (i.e. where my draws could randomly come from group 1 or group 2), what is the variance of that draw" whereas $S_p^2$ asks the question "if I drew a random observation from the population but also recorded whether the observation was from group 1 or from group 2, how much variance would the prediction error of that draw be if I took the mean from the respective group as my prediction?"

One other thing to add. Because $S_p^2$ and $S_c^2$ are related via a variance decomposition and variances can only be non-negative, this implies that (unless $n_1, n_2$ are very small), we should expect that to the extent that the two numbers are different, $S_c^2$ should be larger.

$\endgroup$
13
  • 1
    $\begingroup$ What I wanted to say is that "combined variance" and "pooled variance" are used interchangeably. OP's "combined variance" is also someone's "pooled variance". I am sure the two expressions are not equal; they are not supposed to be equal since they are used in different contexts. $\endgroup$ Commented Dec 17, 2021 at 18:56
  • $\begingroup$ @stats_model, Thank you so much for your clear explanation. However I am not still sure when use $S_c^2$ or $S_p^2$. Let's say, I have 4 groups of independent samples with 5,4,3,6 size. If I want to make the overall mean and Variance then make a 95% CI for overall mean, should I consider $S_p^2$ ? Thanks again $\endgroup$
    – Ati
    Commented Dec 17, 2021 at 21:51
  • $\begingroup$ The answer is unfortunately "it depends". The world where what you call $S_p^2$ is correct is one where the size of the samples is fixed ex ante. The world where $S_c^2$ makes sense is one where your samples are drawn i.i.d. from some meta-population, and which group a given observation ends up in is by chance. $\endgroup$ Commented Dec 17, 2021 at 23:45
  • 1
    $\begingroup$ As explained in my answer, the pooled variance gives an answer to a very specific (and to your point, in many cases not necessarily conceptually useful!) question: what is the mean squared error of the best predictor of $Y$ if all you know is group identity. For instance, pooled variance divided by combined variance is nearly equivalent to $1-R^2$ of a regression of $Y$ on indicators for group identity. $\endgroup$ Commented Dec 15, 2023 at 0:00
  • 1
    $\begingroup$ One example where "average" variance reduction is a useful statistic though, is the context of regression adjustment. In that context, if you control for group identity optimally when analyzing data from an experiment, the factor by which you shrink the variance of the resultant estimator is essentially pooled variance divided by combined variance. See, e.g., nber.org/papers/w30756 $\endgroup$ Commented Dec 15, 2023 at 0:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.