4
$\begingroup$

I would like to estimate the standard deviation of a population. The data I have is the size and mean of each of $k$ independent uniform subsamples: $(n_1, \mu_1), (n_2, \mu_2), \ldots, (n_k, \mu_k)$. Here, the $i$'th subsample has $n_i$ elements, and those $n_i$ elements have a mean of $\mu_i$.

What is the best way to utilize all $2k$ values to produce the best estimate of the population standard deviation $\sigma$?

$\endgroup$
0

1 Answer 1

5
$\begingroup$

Let $\mu$ be the population mean and $\sigma$ be the population SD. You have observations $x_i$ of $k$ independent variables $X_i$ having a common mean $\mu$ and variances $\sigma^2/n_i,$ $i=1,2,\ldots, k.$

You are noncommittal about the sense of "best." To make this specific, let us find the Least Squares estimator of $\sigma.$ This is a Weighted Least Squares problem with weights $1/(1/n_i)=n_i,$ whence

$$\hat\mu = \frac{1}{n_1+n_2+\cdots+n_k}\, \left(n_1x_1 + n_2x_2 + \cdots + n_k x_k\right)$$

with residuals

$$r_i = x_i - \hat\mu$$

which entails

$$\hat\sigma^2 = \frac{1}{k-1}\, \left(n_1 r_1^2 + n_2 r_2^2 + \cdots + n_k r_k^2\right).$$

We may therefore take

$$\hat \sigma = \sqrt{\hat\sigma^2}$$

as a "best" estimate of $\sigma.$

As a check, straightforward algebra will verify that $E[\hat\mu] = \mu$ and $E[\hat\sigma^2]=\sigma^2,$ standard properties of Least Squares estimates. As another check, these calculations reproduce exactly the output of the lm function in R using the $n_i$ in its weights argument.

$\endgroup$
5
  • 1
    $\begingroup$ Thanks, this makes a lot of sense. Btw, I think you have an extra subscript in your residual equation. $\endgroup$
    – dshin
    Commented Apr 5, 2020 at 15:44
  • $\begingroup$ Thank you for the careful review! I have fixed that typo. $\endgroup$
    – whuber
    Commented Apr 5, 2020 at 17:31
  • $\begingroup$ I found that this question appears to have been asked previously, with the top answer coming from you!: stats.stackexchange.com/q/24936/2221 Should this question be removed since it's a repeat? Or is there something sufficiently unique about this question to warrant keeping them both? $\endgroup$
    – dshin
    Commented Apr 6, 2020 at 0:11
  • $\begingroup$ That's a good find (as you might imagine, I had totally forgotten about it). Maybe the best resolution would be for me to add this post as an edit to my answer there. What do you think? $\endgroup$
    – whuber
    Commented Apr 6, 2020 at 12:41
  • $\begingroup$ Sure, would you like to do that? I think this question/answer is a little cleaner so I think there is value to having it. $\endgroup$
    – dshin
    Commented Apr 6, 2020 at 12:46

Not the answer you're looking for? Browse other questions tagged or ask your own question.