# Wrong variance of mean

I'm told that the $Var(\overline{x}) \approx Var(X)/N$, where $\overline{x}$ is the mean of $N$ elements from $X$. However, I have a data set where these variances seem to differ by a factor of about 30.

A series of Bernoulli trials is run. The input data consists of numbers of successes and attempts made in an hour. I have some $400000$ trials unevenly distributed in around $1000$ hours. I take $M$ random samples of hours, and calculate their means as sum of successes divided by sum of attempts. I expect that if $E$ is the mean of these $M$ means, then their variance will be $E(1-E)/\text{sample size}$, however the real variance is found to be much bigger.

I suspect this effect in large part due to the uneven distribution of trials in hours and the way I sample them. Is that true? If so, can I correct for it? I tried aggregating hours with few events, but it didn't seem to help. Could something else be causing it? Should I expect the real variance to be correct if I use all the data rather than random samples?

• Your first sentence is true only if the $X_i$'s are independent. – StatsStudent Feb 25 '15 at 7:36
• Can you clarify the part about hours and uneven distribution. Do your samples have different numbers of trials? If so, how do you obtain $N$ for your formula? – Juho Kokkala Feb 25 '15 at 7:59
• @JuhoKokkala, there could be anywhere from 0 to 20000 trials per hour though most have under 5000. The sample sizes are fairly normally distributed with mean of 100000 and standard deviation of 20000. I take N to be the mean of sample sizes. In this particular sampling the samples do overlap, though I've tried sampling without overlap too without much difference in the ratio of variances. – Karolis Juodelė Feb 25 '15 at 9:20
• Is each individual Bernoulli trial characterized by the same probability of success? – Alecos Papadopoulos Feb 25 '15 at 10:17
• This is called overdispersion, search this site. There can be many reasons. We really need much more details and context of your cae. – kjetil b halvorsen Aug 2 '17 at 13:12