1
$\begingroup$

I have one sample of only 86 values, which is not normally distributed according to the Shapiro-Wilk normality test. Can I still use this formula/code (sorry R code) to estimate the 95% confidence interval?

me <- qt(.975, length(sample) - 1) * sd(sample)/sqrt(length(sample))
lower <- mean(sample) - me
upper <- mean(sample) + me

I would think that it is OK, as the sample mean should be normally distributed according to the central limit theorem? As far as I understand the central limit theorem, the means of several samples of sufficient size (30?) should be normally distributed, even if the distribution is, for example, multi modal. Now what confused me about this scenario is that I only have 1 sample. The function qt is the t-distribution's quantile - in this case for the 95% percentile. This should deal with the smallish sample size and the non normality?

I appreciate that this is basic stuff for you experts, but any feedback would be appreciated. Thanks!

PS:

I am aware of bootstrapping but would like to use "classical statistics" to estimate the CI.

This is related but there is no accepted answer. Here Ben's answer suggests that I am "right". Tony Ladson's answer looks also interesting. Should I use this approach and if so do I have to test for log normality?

Improve this question
$\endgroup$
2
  • $\begingroup$ Do you understand why the confidence interval procedure works for data you know to be normally distributed, even though you only have one sample? $\endgroup$
    – Dave
    Commented Jan 6, 2021 at 15:41
  • $\begingroup$ well I think this is exactly what I struggle with. The confidence interval's width depends on the sample size and I do not know the population standard deviation. I think the t distribution is used because I cannot assume normality and I have a smallish sample size. I can only use the standard normal distribution when I know the population standard deviation AFIK. $\endgroup$
    – cs0815
    Commented Jan 6, 2021 at 16:16

1 Answer 1

Reset to default
1
$\begingroup$

You could try a non-parametric bootstrap:

set.seed(1976)
N <- 86
sample <- c(rexp(floor(N/2), 5), rexp(ceiling(N/2), 1/10))
hist(sample)

length(sample)
mean(sample)
# expected
(floor(N/2)*1/5 + ceiling(N/2)*10) / N

me <- qt(.975, length(sample) - 1) * sd(sample)/sqrt(length(sample))
lower <- mean(sample) - me
upper <- mean(sample) + me
c(lower, upper)

# bootstrap
require(boot)
b <- boot(sample, statistic = function(x, w) mean(x[w]), R = 10000, sim = "ordinary")
b

boot.ci(b, conf = 0.95, type = "perc")

plot(b)
```
Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks - sorry I forgot to mention - I am aware of bootstrapping and it is an option but I do not want to use bootstrapping - sorry $\endgroup$
    – cs0815
    Commented Jan 6, 2021 at 16:05
  • 1
    $\begingroup$ Perhaps you could explain why you don't want to use a bootstrap? A bootstrap is a distribution-free technique that is always applicable. It is possible that your sample is so skewed that 86 observations are insufficient to ensure the CLT gives you a t-distribution. We can't tell without your sample. In the absence of that specific information, non-parametric methods should be your next option. $\endgroup$
    – R Carnell
    Commented Jan 6, 2021 at 16:10
  • $\begingroup$ Thanks. I would think that n >> 30 would allow me to use the t distribution? $\endgroup$
    – cs0815
    Commented Jan 6, 2021 at 16:11
  • 1
    $\begingroup$ That is the thumbrule, but the thumbrule is not universal. In my experience, some distributions, like the distribution of account balances in a bank, are so skewed that 400 samples are required to ensure the distribution of the sample mean is t distributed (obviously at a certain tolerance). If you are just asking about the thumbrule, you've got it. If you are asking if 86 is enough in practice, it depends on your sample distribution. $\endgroup$
    – R Carnell
    Commented Jan 6, 2021 at 16:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.