3
$\begingroup$

I have a population of 7954 subjects and multiple random samples from it:

  • 1st sample size 2,665 with a standard deviation 1.386 and mean -7.45
  • 2nd sample size 2,399 with std dev 1.53 and mean -7.33
  • 3rd sample size 802 with std dev 3.02 and mean -7.45.

I want to calculate a standard deviation for the population. Also, if I take another sample of 2,133 what can be my standard deviation be for it?

σ = s * √(n / (n - 1)) and σₓ = σ/√n I couldn't yield any results or validate them.

$\endgroup$
11
  • $\begingroup$ Welcome to Cross Validated! It seems like you would have to measure some subjects more than once, meaning that you would not be able to get the exact population standard deviation like you would if you observed all $7954$ subjects and did the calculation. Is this correct? $\endgroup$
    – Dave
    Commented Feb 10, 2023 at 17:48
  • $\begingroup$ the only way to get the population metric is to measure it on the population $\endgroup$
    – Aksakal
    Commented Feb 10, 2023 at 17:51
  • $\begingroup$ @Aksakal I do not agree. If you divide the $7954$ subjects into multiple groups and calculate the means, variances, and sample sizes for each group, those can be combined to give the values you would have gotten if you calculated them on the entire group. That some subjects in the OP might be omitted or double-counted complicates this, however. $\endgroup$
    – Dave
    Commented Feb 10, 2023 at 17:59
  • $\begingroup$ @Dave, you describe how you measure the population in batches. that's not what OP is doing $\endgroup$
    – Aksakal
    Commented Feb 10, 2023 at 18:04
  • 2
    $\begingroup$ @whuber they are with replacement. $\endgroup$ Commented Feb 10, 2023 at 20:59

1 Answer 1

3
$\begingroup$

Taking this question (and comments) at face value, it concerns a distribution $F$ (the empirical distribution of a population) and statistics from multiple independent random samples of $F.$ This formulation is possible because the sampling was done with replacement (as stated in a comment to the question).


To establish a notation, let the samples be indexed by $i = 1, 2, 3$ (the results will obviously generalize to other than three samples); designate their sizes as $n_i;$ let their means and standard deviations be $m_i$ and $s_i,$ respectively; and suppose the observations in sample $i$ are $x_{ij},$ $j = 1, 2, \ldots, n_i.$

Combining the $x_{ij}$ into a single sample yields a random sample with replacement from $F.$ The question asks how to estimate the standard deviation of $F$ from the combined sample.

The answer is an algebraic identity that expressed the formulas for the counts, sums, and sum of squares of the combined sample in terms of the individual sample statistics.

Most likely you used a formula equivalent to

$$m_i n_i = \sum_{j=1}^{n_i} x_{ij}$$

for the means, a relation used to simplify the standard deviation below. This implies the residuals sum to zero,

$$\sum_{j=1}^{n_i} (x_{ij} - m_i) =\sum_{j=1}^{n_i} x_{ij} - \sum_{j=1}^{n_i}m_i = \sum_{j=1}^{n_i} x_{ij} - m_i n_i = 0,$$

which will also be exploited next.

Your formula for the standard deviations probably was something like

$$\begin{aligned} (n_i-1)s_i^2 &= \sum_{j=1}^{n_i} \left(x_{ij} - m_i\right)^2 \\ &=\sum_j x_{ij}(x_{ij} -m_i) - \sum_j m_i(x_{ij} - m_i)\\ &= \sum_j (x_{ij}^2 -x_{ij}m_i) - m_i\sum_j (x_{ij} - m_i)\\ &=\sum_j x_{ij}^2 - m_i \sum_j x_{ij} - m_i(0)\\ &= \sum_{j=1}^{n_i} x_{ij}^2 - n_i m_i^2. \end{aligned}$$

We thereby obtain an expression for the sum of squares of all the data in terms of the means and standard deviations of the individual samples:

$$\sum_{j=1}^{n_i} x_{ij}^2 = (n_i-1)s_i^2 + n_i m_i^2.$$

  • The combined count is $$n = \sum_{i=1}^3 n_i.$$

  • The combined sum is $$S = \sum_{i=1}^3 n_i m_i.$$

  • The combined sum of squares is$$SS = \sum_{i=1}^3 (n_i-1)s_i^2 + n_i m_i^2.$$

Thus, the combined statistics are

$$m = \frac{S}{n}$$ and

$$s^2 = \frac{1}{n-1}(SS - nm^2).$$


Remarks

$s^2$ is an unbiased estimator of the population variance. Thus, on average, any random sample (of any size greater than $1$) will have a variance equal to $s^2.$ This implies its standard deviation will, on average, not equal $s,$ but it ought to be close. Without more detailed information about the data, this is the best we can do.

If instead you used different formulas for the $m_i$ and $s_i,$ you can work through the algebra in the same way. You won't be successful with some formulas (such as those based on order statistics), but with the commonest ones the algebra will be equally simple.

Statistics based on higher moments can be combined with the same technique. For instance, various formulas for skewness involve the counts, sums, sums of squares, and sums of cubes of the data. Solve for these sums in terms of the counts, means, standard deviations, and skewnesses of the individual samples; combine them; and apply the desired formula.

$\endgroup$
8
  • $\begingroup$ I'm curious about the case where we have the mean / std for each group, but we don't know the sample size. Is there a reasonable way to estimate a combined mean/std in this case? My thought was to take the limit as all n_i -> inf, which causes a nice simplification. I assume the answer is: it depends? $\endgroup$
    – Erotemic
    Commented Mar 19, 2023 at 2:48
  • 1
    $\begingroup$ @Erotemic Without sample size information, the best you can hope for would be to establish a range of possible results. Sample sizes are important when combining most statistics because the sample sizes are so closely related to the uncertainties in those statistics. Because of this, it's a good idea to distrust any statistic for which you do not know the sample size. $\endgroup$
    – whuber
    Commented Mar 19, 2023 at 13:24
  • $\begingroup$ That's good advice. Let me try and focus in on why I'm asking. I have mean/std statistics computed on several different geographic regions (satellite images). I have a large imbalance of observations, so my sample size for some regions is much larger than others. But I want to combine them in a "macro average"-like way. My thought would be to pretend the number of obs are the same in some way. I do worry about differences in uncertainties, but I don't want to bias towards larger regions. The number of obs is always "large"; is there a standard practice here? $\endgroup$
    – Erotemic
    Commented Mar 21, 2023 at 18:05
  • $\begingroup$ @Erotemic Yours might be a complex problem, because if by "satellite images" you mean the pixel values in those images, it is rare indeed that those values can be analyzed as if they were statistically independent. $\endgroup$
    – whuber
    Commented Mar 21, 2023 at 20:26
  • $\begingroup$ Yes, this is probably not the appropriate place to discuss specifics. But just to complete the thread: I'm normalizing input data to a neural network so after (x - μ) / σ on average the network sees μ=0, σ=1. Regions have different μ,σ,n, but I need a single global μ,σ. I want a "macro average" so regions are considered equally, so I'm debating between n->∞, or choosing constant n. $\endgroup$
    – Erotemic
    Commented Mar 22, 2023 at 16:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.