5
$\begingroup$

I have a bimodal distribution, and I wish to estimate the mean and standard deviation of the population (well, these could be 2 sub-populations according to the shape).

With the mean I have no problem, the sample is relatively large (n~70), and so the sample mean is approximately normally distributed according to the central limit theorem. I can calculate the corresponding 95% CI.

Now I want to do the same for the standard deviation, but I am not sure if the same theory apply. Can I say that the variance is approximately chi squared distributed when the sample is large ?

Edit:

To test my question, I ran this simulation in R:

mu1 <- 23   
mu2 <- 40
sig1 <- 4.3
sig2 <- 2.8
cpct <- 0.32
m <- 10000
means <-rep(0,m)
sds <-rep(0,m)
vars <-rep(0,m)

bimodalDistFunc <- function (n,cpct, mu1, mu2, sig1, sig2) {
  y0 <- rnorm(n,mean=mu1, sd = sig1)
  y1 <- rnorm(n,mean=mu2, sd = sig2)

  flag <- rbinom(n,size=1,prob=cpct)
  y <- y0*(1 - flag) + y1*flag 
}


for (i in 1:m)
{
    bimodalData <- bimodalDistFunc(n=10000,cpct,mu1,mu2, sig1,sig2)
    means[i] = mean(bimodalData)    
    sds[i] = sqrt(var(bimodalData))
    vars[i] = var(bimodalData)
}

par(mfrow=c(2,2))
hist(bimodalData)
hist(means)
hist(sds)
hist(vars)

And I got the following result:

enter image description here

The distribution of the variances is not chi squared, but why is it normal ? (the bimodal histogram is an example of one of the simulated distributions)

$\endgroup$
3
$\begingroup$

This won't be rigorous, but it should give you a feel for why it might often tend to occur:

1) Imagine you were calculating not the $n-1$ denominator variance, but the $n$-denominator version (this only gives a scaling factor, so it doesn't impact the shape you see... and that scaling factor goes to 1 in the limit)

2) Consider that as sample sizes become large, the distribution of $X_i-\overline{X}$ approaches the distribution of $X_i-\mu$ (e.g. via Slutsky's theorem).

3) Now consider $Y=(X_i-\mu)^2$; by the Central Limit theorem $\sqrt{n}(\overline{Y}-E(Y))$ converges to a normal distribution, as long as the conditions hold (e.g. you need $\text{Var}(Y)$ to exist). Further note that $E(Y)=\sigma^2$.

So - in essence because the sample variance is effectively just a kind of average - you might not be surprised to see sample variance to approach normality (centered at the population variance) as sample sizes become large.

In Asymptotic Statistics, A. W. van der Vaart pursues a somewhat more rigorous argument (see end p26-p27) by writing the $n$-denominator form of variance as a function of $\overline{X}$ and $\overline{X^2}$ and looking at the multivariate CLT applied to the pair.


However, you have to be careful - in some situations at least - this approximate normality doesn't hold. All the conditions must hold for the argument to go through. (In particular, there's something a little unusual about $(\overline{X}, \overline{X^2})$ for the Bernoulli($\frac{1}{2}$) case.)

Consider the Bernoulli distribution with $p=\frac{_1}{^2}$; for that case, as sample sizes become large, sample variance doesn't approach normality.

enter image description here

In that case, at least, the distribution of the sample variance - even in quite large samples - doesn't become more normal.

$\endgroup$
  • $\begingroup$ Thank you. So how would you make inference on the standard deviation given a bimodal distribution like the one in the image? I can get two standard deviations out of it, one for each distribution within the bimodal, but is there a way to pool them together and get a confidence interval ? $\endgroup$ – user52407 Jul 21 '14 at 6:45
  • $\begingroup$ There are several possible options. If you model the bimodal distribution (say as a mixture of unimodal distributions), you might be able to derive results for inference on the variance or standard deviation. If the samples are large enough, you can still apply the result of van der Vaart, if you know the population mean and variance of the sample variance, or if the samples are large enough that you can act as if Slutsky's theorem was to "kick in"...(ctd) $\endgroup$ – Glen_b -Reinstate Monica Jul 21 '14 at 9:32
  • $\begingroup$ (ctd) ... or see some of the details here. Finally, you might try simulation or bootstrapping. $\endgroup$ – Glen_b -Reinstate Monica Jul 21 '14 at 9:33
  • $\begingroup$ Also see here $\endgroup$ – Glen_b -Reinstate Monica Jul 21 '14 at 10:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.