# How do I calculate confidence intervals for a non-normal distribution?

I have 383 samples that have a heavy bias for some common values, how would I calculate the 95% CI for the mean? The CI that I calculated seems way off, which I assume is because my data does not look like a curve when I make a histogram. So I think I have to use something like bootstrapping, which I don't understand very well.

• One solution would be to use the asymptotic CI that makes use of the fact that the RV $\frac{\bar{X}-\mu}{S/\sqrt{n}}$ has a limiting standard normal distribution. Your sample is reasonably large so it might make for a good approximation. – JohnK Aug 21 '14 at 20:19
• No, you will find really bad tail coverage in both tails of the confidence interval using that approach. The average coverage might by good luck be OK but both tail error rates will be wrong. – Frank Harrell Aug 21 '14 at 20:49
• What does "heavy bias for some common values" mean? Note that bias has a particular meaning in statistics; you should try to avoid it if you don't mean that. Do you simply mean "some particular values occur very often"? Can you show your calculations and some display or table of your data? – Glen_b Aug 21 '14 at 23:53
• There is a good discussion in this paper Wang ( 2001) Confidence interval for the mean of non-normal data Quality and reliability engineering international 17: 257-267 – Tony Ladson Feb 4 '16 at 1:03

## 4 Answers

Yes, bootstrap is an alternative for obtaining confidence intervals for the mean (and you have to make a bit of effort if you want to understand the method).

The idea is as follows:

1. Resample with replacement B times.
2. For each of these samples calculate the sample mean.
3. Calculate an appropriate bootstrap confidence interval.

Concerning the last step, there are several types of bootstrap confidence interval (BCI). The following references present a discussion on the properties of different types of BCI:

http://staff.ustc.edu.cn/~zwp/teach/Stat-Comp/Efron_Bootstrap_CIs.pdf

http://www.tau.ac.il/~saharon/Boot/10.1.1.133.8405.pdf

It is a good practice to calculate several BCI and try to understand possible discrepancies between them.

In R, you can easily implement this idea using the R package 'boot' as follows:

rm(list=ls())
# Simulated data
set.seed(123)
data0 = rgamma(383,5,3)
mean(data0) # Sample mean

hist(data0) # Histogram of the data

library(boot)

# function to obtain the mean
Bmean <- function(data, indices) {
d <- data[indices] # allows boot to select sample
return(mean(d))
}

# bootstrapping with 1000 replications
results <- boot(data=data0, statistic=Bmean, R=1000)

# view results
results
plot(results)

# get 95% confidence interval
boot.ci(results, type=c("norm", "basic", "perc", "bca"))

• The last step, calculate several, implies fishing for a CI you like from the result. You should decide on the kind of CI you want based on what they are beforehand. – John Aug 21 '14 at 20:35
• Absolutely, but without such a description of the reason in your answer it "implies" fishing. And you're still not stating that it's critical to actually pick the CI you want first. I'm suggesting an update of the answer with some critical information for a naive questioner. It would be even better if you state which CI you generally prefer and why, or which one you prefer in cases like this one and why. – John Aug 21 '14 at 20:46
• In this Khan Academy video:link He seems to be using a non-normal distribution, but uses "Sampling distribution of the sample mean" to get a normal distribution. He's basically creating samples of various sizes and demonstrating that it forms a bell curve. Is that basically a crude form of bootstrapping? or something entirely different, because it seems to get me the results I'm after. – IhaveCandy Aug 21 '14 at 21:47
• @IhaveCandy: No. It demonstrates the Central Limit Theorem, i.e. how the sampling distribution of the mean tends to the normal, even for values following a very "unnormal" distribution. That is why the simple z confidence interval won't be much different to any other fancy solution, e.g. bootstrap. – Michael M Aug 22 '14 at 15:16
• @IhaveCandy Please see my comment above, Michael Mayer is making the same point. – JohnK Aug 22 '14 at 16:53

Another standard alternative is to calculate the CI with the Wilcoxon test. In R

wilcox.test(your-data, conf.int = TRUE, conf.level = 0.95)


Unfortunately, it gives you the CI around the (pseudo)median not the mean, but then if the data is heavily non-normal maybe the median is a more informative measure.

For log-normal data, Olsson (2005) suggests a 'modified Cox method'

If $X$ is log-normally distributed and $\rm{E}(X) = \theta$, the a confidence interval for $\log(\theta)$ is:

$$\bar{Y} = \frac{S^2}{2} \pm t_{df}\sqrt{\frac{S^2}{n} + \frac{S^4}{2(n-1)} }$$

Where $Y = \log(X)$, the sample mean of $Y$ is $\bar{Y}$ and the sample variance of $Y$ is $S^2$. For df, use n-1.

An R function is below:

ModifiedCox <- function(x){
n <- length(x)
y <- log(x)
y.m <- mean(y)
y.var <- var(y)

my.t <- qt(0.975, df = n-1)

my.mean <- mean(x)
upper <- y.m + y.var/2 + my.t*sqrt(y.var/n + y.var^2/(2*(n - 1)))
lower <- y.m + y.var/2 - my.t*sqrt(y.var/n + y.var^2/(2*(n - 1)))

return(list(upper = exp(upper), mean = my.mean, lower = exp(lower)))

}


Repeating the example from Olsson's paper

CO.level <- c(12.5, 20, 4, 20, 25, 170, 15, 20, 15)

ModifiedCox(CO.level)
$upper  78.72254$mean
 33.5

\$lower
 12.30929

• is there a test for log normality? – cs0815 Jan 6 at 14:47
• try goft::lnorm.test – Tony Ladson Jan 9 at 5:25

You can just use a standard confidence interval for the mean: Bear in mind that when we calculate confidence intervals for the mean, we can appeal to the central limit theorem and use the standard interval (using the critical points of the T-distribution), even if the underlying data is non-normal. In fact, so long as the distribution of the underlying data has finite variance, the distribution of the sample mean with $$n=383$$ observations should be virtually indistinguishable from a normal distribution. This will be the case even if the underlying distribution of the data is extremely different to a normal distribution.