I'm working on my master thesis, and something's come up where I don't know if I'm "allowed" to do this and call it good science, I've scoured the internet to no avail of finding my answer, so I figured I'd ask you.

In short, I have a certain unknown distribution generating data, I'm trying to determine this distribution's $\sigma$ (I actually don't care about $\mu$). So what I'm attempting to do is to sample the data, and use the CLT to my advantage for this task. The thing is, these data points aren't sampled in equal size. (and sadly I can't explain why on here, NDA stuff with my master thesis...) Sometimes I'll have 5 points in a sample, sometimes I'll have 10, sometimes 50 (more often than not, it's <15 sample sizes)

My question is this :

Am I allowed to use the CLT with variable sample sizes to determine the unknown distribution's $\sigma$?

Everywhere I look, I see that $\frac{\sigma}{\sqrt{n}}$ shows up as the sampling distribution's standard deviation, but since my $n$ is not constant, I don't know what to do.

I've actually done a Monte Carlo experiment where I sample a known distribution with random sample sizes (sample size as a RV following an exponential distribution) and the sample distribution's $\sigma$ gets close to the known distribution's $\sigma$ but not quite...

Any help is appreciated. Thanks in advance!

  • $\begingroup$ The CLT has to do with convergence to normality. Your issue is to get a combined variance estimate from samples of different sizes. $\endgroup$ – BruceET Sep 24 at 10:16
  • $\begingroup$ @BruceET I'll look into variance estimate, thanks for that! I was actually hoping to "trick" my way to the variance estimate using the CLT, since its quite convenient to calculate $\endgroup$ – big_skapinsky Sep 24 at 10:39
  • $\begingroup$ @BruceET that actually helped a LOT! I researched further and found about "pooled variance" which seems to be exactly what I was searching for. Thanks for the hint! $\endgroup$ – big_skapinsky Sep 24 at 11:25
  • $\begingroup$ Good. My $\hat \sigma^2$ is sometimes called a 'pooled variance'. (Especially, in 'a pooled 2-sample t test'. If this is the answer you need, then please check to 'Accept' to remove this Q from list of unanswered questions. $\endgroup$ – BruceET Sep 24 at 18:07

Assuming that all samples are from populations with the same variance $\sigma^2,$ you can estimate the variance as

$$\hat\sigma^2 = \frac{\sum_{i=1}^n (r_i - 1)S_i^2}{\sum_{i=1}^n r_i\; - n},$$ where you have $n$ samples with $r_i\ge 2, i=1,\dots, n$ replications in each, and $S_i^2$ is the sample variance of the $r_i$ replications in sample $i.$ That is $S_i^2 = \frac{1}{r_i - 1}\sum_{j=1}^{r_i}(X_{ij} - \bar X_i)^2$ and $\bar X_i = \frac{1}{r_i}\sum_{j=1}^{r_i}X_{ij}.$

If data are normal, then the relationship $\frac{(\sum_{i=1}^n r_i\; - n)\hat\sigma}{\sigma^2} \sim \mathsf{Chisq}(\nu),$ with $\nu = \sum_{i=1}^n r_i\; - n,$ can be used to make a confidence interval for $\sigma^2.$

The displayed formula is essentially the formula for the denominator mean square in an unbalanced one-way ANOVA with $n$ levels of the factor (groups) and $r_i$ replications per factor (group), where all groups are from populations with the same variance $\sigma^2.$ [It is a weighted average of the individual sample variances, where weights are degrees of freedom $r_i-1,$ not numbers $r_i$ of replications.]

| cite | improve this answer | |
  • $\begingroup$ I'm a bit confused (still not entirely 100% proficient in the statistics jargon, forgive me) But in my case, using your equation gives me a negative denominator... n can get up to the hundreds, but r_i rarely goes above 50. Or am I not interpreting these correctly? Thanks for your help!! $\endgroup$ – big_skapinsky Sep 24 at 10:45
  • $\begingroup$ Negative denom is impossible. In displayed eqn denom is $\sum_i r_i\, -n = (\sum_i r_i) -n = \sum_i (r_i-1).$ Amounts to grand total sample size minus nr of groups. // Also, if you are generating $r_i$ at random, then make sure $r_i\ge 2.$ $\endgroup$ – BruceET Sep 24 at 18:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.