# How do I calculate confidence intervals for a non-normal distribution?

I have 383 samples that have a heavy bias for some common values, how would I calculate the 95% CI for the mean? The CI that I calculated seems way off, which I assume is because my data does not look like a curve when I make a histogram. So I think I have to use something like bootstrapping, which I don't understand very well.

• One solution would be to use the asymptotic CI that makes use of the fact that the RV $\frac{\bar{X}-\mu}{S/\sqrt{n}}$ has a limiting standard normal distribution. Your sample is reasonably large so it might make for a good approximation. Commented Aug 21, 2014 at 20:19
• No, you will find really bad tail coverage in both tails of the confidence interval using that approach. The average coverage might by good luck be OK but both tail error rates will be wrong. Commented Aug 21, 2014 at 20:49
• What does "heavy bias for some common values" mean? Note that bias has a particular meaning in statistics; you should try to avoid it if you don't mean that. Do you simply mean "some particular values occur very often"? Can you show your calculations and some display or table of your data? Commented Aug 21, 2014 at 23:53
• There is a good discussion in this paper Wang ( 2001) Confidence interval for the mean of non-normal data Quality and reliability engineering international 17: 257-267 Commented Feb 4, 2016 at 1:03

Yes, bootstrap is an alternative for obtaining confidence intervals for the mean (and you have to make a bit of effort if you want to understand the method).

The idea is as follows:

1. Resample with replacement B times.
2. For each of these samples calculate the sample mean.
3. Calculate an appropriate bootstrap confidence interval.

Concerning the last step, there are several types of bootstrap confidence interval (BCI). The following references present a discussion on the properties of different types of BCI:

http://staff.ustc.edu.cn/~zwp/teach/Stat-Comp/Efron_Bootstrap_CIs.pdf

http://www.tau.ac.il/~saharon/Boot/10.1.1.133.8405.pdf

It is a good practice to calculate several BCI and try to understand possible discrepancies between them.

In R, you can easily implement this idea using the R package 'boot' as follows:

rm(list=ls())
# Simulated data
set.seed(123)
data0 = rgamma(383,5,3)
mean(data0) # Sample mean

hist(data0) # Histogram of the data

library(boot)

# function to obtain the mean
Bmean <- function(data, indices) {
d <- data[indices] # allows boot to select sample
return(mean(d))
}

# bootstrapping with 1000 replications
results <- boot(data=data0, statistic=Bmean, R=1000)

# view results
results
plot(results)

# get 95% confidence interval
boot.ci(results, type=c("norm", "basic", "perc", "bca"))

• The last step, calculate several, implies fishing for a CI you like from the result. You should decide on the kind of CI you want based on what they are beforehand.
– John
Commented Aug 21, 2014 at 20:35
• Absolutely, but without such a description of the reason in your answer it "implies" fishing. And you're still not stating that it's critical to actually pick the CI you want first. I'm suggesting an update of the answer with some critical information for a naive questioner. It would be even better if you state which CI you generally prefer and why, or which one you prefer in cases like this one and why.
– John
Commented Aug 21, 2014 at 20:46
• In this Khan Academy video:link He seems to be using a non-normal distribution, but uses "Sampling distribution of the sample mean" to get a normal distribution. He's basically creating samples of various sizes and demonstrating that it forms a bell curve. Is that basically a crude form of bootstrapping? or something entirely different, because it seems to get me the results I'm after. Commented Aug 21, 2014 at 21:47
• @IhaveCandy: No. It demonstrates the Central Limit Theorem, i.e. how the sampling distribution of the mean tends to the normal, even for values following a very "unnormal" distribution. That is why the simple z confidence interval won't be much different to any other fancy solution, e.g. bootstrap. Commented Aug 22, 2014 at 15:16
• @IhaveCandy Please see my comment above, Michael Mayer is making the same point. Commented Aug 22, 2014 at 16:53

Another standard alternative is to calculate the CI with the Wilcoxon test. In R

wilcox.test(your-data, conf.int = TRUE, conf.level = 0.95)


Unfortunately, it gives you the CI around the (pseudo)median not the mean, but then if the data is heavily non-normal maybe the median is a more informative measure.

• Pythons scipy.stats.wilcoxon does not have the parameter conf.int and conf.level. Anyone knows how to solve this in Python? Commented Jun 15, 2022 at 8:55

You can just use a standard confidence interval for the mean: Bear in mind that when we calculate confidence intervals for the mean, we can appeal to the central limit theorem and use the standard interval (using the critical points of the T-distribution), even if the underlying data is non-normal. In fact, so long as the data is IID (Independent and Identically Distributed) and the distribution of the data has finite variance, the distribution of the sample mean with $$n=383$$ observations should be virtually indistinguishable from a normal distribution. This will be the case even if the underlying distribution of the data is extremely different to a normal distribution.

• could you please expand the meaning of IID? Commented Aug 3, 2022 at 15:29
• @Andrei: IID is an acronym for independent and identically distributed.
– Ben
Commented Aug 3, 2022 at 22:06
• For a lognormal(0,1) distribution n=50,000 is not large enough to guarantee adequatey accurate confidence intervals. Commented Aug 4, 2022 at 11:27
• I got terrible results for that - see if you can see the source of our differences - code is at hbiostat.org/bbr/htest.html#central-limit-theorem Commented Sep 4, 2022 at 12:15
• @AbhishekDivekar: The T-distribution is used when the variance is unknown (and estimated using the sample variance).
– Ben
Commented Aug 4 at 10:18

For log-normal data, Olsson (2005) suggests a 'modified Cox method'

If $$X$$ is log-normally distributed and $$\rm{E}(X) = \theta$$, the a confidence interval for $$\log(\theta)$$ is:

$$\bar{Y} = \frac{S^2}{2} \pm t_{df}\sqrt{\frac{S^2}{n} + \frac{S^4}{2(n-1)} }$$

Where $$Y = \log(X)$$, the sample mean of $$Y$$ is $$\bar{Y}$$ and the sample variance of $$Y$$ is $$S^2$$. For df, use n-1.

An R function is below:

ModifiedCox <- function(x){
n <- length(x)
y <- log(x)
y.m <- mean(y)
y.var <- var(y)

my.t <- qt(0.975, df = n-1)

my.mean <- mean(x)
upper <- y.m + y.var/2 + my.t*sqrt(y.var/n + y.var^2/(2*(n - 1)))
lower <- y.m + y.var/2 - my.t*sqrt(y.var/n + y.var^2/(2*(n - 1)))

return(list(upper = exp(upper), mean = my.mean, lower = exp(lower)))

}


Repeating the example from Olsson's paper

CO.level <- c(12.5, 20, 4, 20, 25, 170, 15, 20, 15)

ModifiedCox(CO.level)
$upper [1] 78.72254$mean
[1] 33.5

\$lower
[1] 12.30929

• is there a test for log normality? Commented Jan 6, 2021 at 14:47
• try goft::lnorm.test Commented Jan 9, 2021 at 5:25