2
$\begingroup$

As someone with quite a weak understanding of statistics, I'd like to know:

Is it possible to calculate confidence intervals for a population range in general from a sample?

What I have learned so far is that I can calculate confidence intervals for an estimated population mean from a sample (where the mean and standard deviation is unknown) by using the t-distribution or normal distribution depending on my sample size using formulas like (where $S$ is the estimated population standard deviation of the sample, Bessel-corrected, and $n$ is the sample size):

$$\bar{X} \pm t^*\frac{S}{\sqrt{n}}$$

And I'd use the 'regular' normal distribution if somehow the population standard deviation was known:

$$\bar{X} \pm z\frac{\sigma}{\sqrt{n}}$$

But this is all just for the population mean. What if I wanted to obtain a confidence interval whether a particular data point is part of the population or an outlier? For example, let's say I'm examining the length of dog tails, and I'm not interested in getting an accurate range for the average dog tail length, but rather if a specific tail length can be a dog's measurement or not. So I basically want a range or confidence interval that would be much larger, and then, when I have a measurement that lies outside this range (e.g. my dog tail range would be 5cm $\pm$ 4cm, but I get a measurement of 14cm; yes, the numbers are unrealistic), I can say something like "with 95% accuracy, this measurement is not from a dog". And this would then be different from the confidence interval for the mean of the dog tail length, which could be something like 5cm $\pm$ 1cm, i.e. much more precise, of course.

Naively, I would believe that I can just use the sample mean (estimated population mean) and estimated population standard deviation for that like this:

$$\bar{X} \pm z \cdot S$$

Would this be a correct procedure or completely wrong? Is it even possible to calculate a confidence interval for the whole population range (for outlier detection, for example) or is this confidence interval method just not suitable for that? If so, how would one define or calculate a range for the whole population?

$\endgroup$
2
  • 1
    $\begingroup$ What you are describing is not a confidence interval but a prediction interval. $\endgroup$ – Frans Rodenburg Nov 13 '19 at 12:29
  • $\begingroup$ Frans is correct, but you could also use a box plot to get a pictorial representation of what are generally considered outliers. You have to use judgement when considering whether an outlier is a real possibility or a measurement error. $\endgroup$ – Robert Jones Nov 13 '19 at 23:40
1
$\begingroup$

You can calculate confidence intervals for a variety of population parameters based on a sample. There may be formulae to calculate some of these. In other cases, a bootstrap procedure can be used.

As @rep_ho points out, your dog tail example is similar to those presented in introductory texts covering hypothesis testing with z-tests and t-tests. You might start by looking at these. (See, for example, the free OpenIntro Statistics book).

This may not be exactly what you are looking for, though. If you are starting with the assumption that dog tail length is normally distributed, there's some probability that a tail of any length might be found. Just to make up some numbers (though you could calculate these from the z-score or t value): There's a 50% chance of getting a tail longer than 5 cm; A 33% chance of getting a tail longer than 8 cm; ... So there may just be a 1/1000 or 1/10000 or 1/1000000 chance of finding a tail of, say, greater than 14 cm.

This is different than having a "range" of acceptable values.

Realistically speaking, you might be interested in a classification process. (e.g. k-means clustering). If we start with many samples of species' tail length and ear length, for an unknown specimen of tail and ear, we can determine if the specimen is likely, e.g. a dog, a rabbit, a cat, and so on.

$\endgroup$
0
$\begingroup$

You don't need CI for that. You can just look how what is the z-score of that new observation, i.e. how much standard deviation form the mean the new observation is. Then you have to decide what is the appropriate threshold to use to claiming that something is an outlier.

$\endgroup$
0
$\begingroup$

Intervals are for anything. You could have an uncertainty in the uncertainty.

Here is an answer that evaluates the uncertainty around the max: Why use extreme value theory?

Bootstrapping can be useful for finding intervals (or other measures) for things that can be simulated much more easily than they can be symbolically analyzed. Consider the example here: https://www.datacamp.com/community/tutorials/bootstrap-r

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.