Does this single value match that distribution?

Question

this feels like a very naive question but I'm having difficulty seeing the answer.

I have one set of 30 values. Independently I obtained a 31st value. Null hypothesis is that the 31st value is part of the same distribution. Alternative is that its different. I want some kind of p-value or likelihood measure.

Some thoughts I've had:

• This is similar to wanting to do a two-sample t-test - except that for the second sample I only have a single value, and the 30 values aren't necessarily normally distributed.
• If instead of 30 measurement I had 10000 measurement, the rank of the single measurement could provide some useful information.

How can I calculate this likelihood or p-value?

Thanks! Yannick

Answer 1

In the unimodal case the Vysochanskij-Petunin inequality can give you a rough prediction interval. Here is the wikipedia site: http://en.wikipedia.org/wiki/Vysochanski%C3%AF%E2%80%93Petunin_inequality

Using $\lambda = 3$ will result in an approximate 95% prediction interval.

So you estimate the mean and standard deviation of your population and just use the sample mean $\bar x $ plus or minus $3s$ as your interval.

There are a couple of problems with this approach. You don't really know the mean or standard deviation; you are using estimates. And in general you won't have unimodal distributions meaning you will have to use specialized versions of Chebyshev's inequality. But at least you have a starting point.

For the general case, Konijn (The American Statistician, February 1987) states the order statistics may be used as a prediction interval. So $ \left[ x_{(i)},x_{(j)} \right]$ is a prediction interval for $X$ with what Konijn calls size ${{j-i} \over {n+1}}. $ Size is defined as "the greatest lower bound (with regard to the set of joint distributions that are admitted) of the probability that the interval will cover the value that $X$ is to take on." With this approach a 93.6% prediction interval would be $ \left[ x_{(1)},x_{(30)} \right].$

He also gives an approach attributed to Saw, Yang, and Mo: $$\left[ \bar x -\lambda \left(1 + {1 \over n}\right)^{1/2}s \ , \ \bar x + \lambda \left(1 + {1 \over n}\right)^{1/2}s \right],$$ with details on the coverage given in the article.

For example with $n=30,$ using $\lambda = 3.2$ would give coverage exceeding 90%.

Answer 2

Some thoughts I've had:

This is similar to wanting to do a two-sample t-test - except that for the second sample I only have a single value, and the 30 values aren't necessarily normally distributed.

Correct. The idea is a bit like a t-test with a single value. Since the distribution is not known, and normality with only 30 data points may be a bit hard to swallow, this calls for some kind of non-parametric test.

If instead of 30 measurement I had 10000 measurement, the rank of the single measurement could provide some useful information.

Even with 30 measurements the rank can be informative.

As @whuber has pointed out, you want some kind of prediction interval. For the non-parametric case, what you are asking, essentially, is the following: what is the probability that a given data point would have by chance the rank we observe for your 31st measurement?

This can be addressed through a simple permutation test. Here's an example with 15 values and a novel (16th observation) that is actually larger than any of the previous:

932
915
865
998
521
462
688
1228
746
433
662
404
301
473
647

new value: 1374

We perform N permutations, where the order of the elements in the list is shuffled, then ask the question: what is the rank for the value of the first element in the (shuffled) list?

Performing N=1,000 permutations gives us 608 cases in which the rank of the first element in the list is equal or better to the rank of the new value (actually equal, since the new value is the best one). Running the simulation again for 1,000 permutations, we get 658 such cases, then 663...

If we perform N=1,000,000 permutations, we obtain 62825 cases in which the rank of the first element in the list is equal or better to the rank of the new value (further simulations give 62871 cases, then 62840...). If the take the ratio between cases in which the condition is satisfied and total number of permutations, we get numbers like 0.062825, 0.062871, 0.06284...

You can see these values converge towards 1/16=0.0625 (6.25%), which as @whuber notes, is the probability that a given value (out of 16) drawn at random has the best possible rank among them.

For a new dataset, where the new value is the second best value (i.e. rank 2):

6423
8552
6341
6410
6589
6134
6500
6746
8176
6264
6365
5930
6331
6012
5594

new value: 8202

we get (for N=1,000,000 permutations): 125235, 124883... favorable cases which, again, approximates the probability that a given value (out of 16) drawn at random has the second best possible rank among them: 2/16=0.125 (12.5%).