# Probability that a set of values came from a distribution

I have a probability distribution. And I have a set of values. I need to figure out how to calculate the probability that these values were generated by the same model as the distribution.

I found this answer: Probability that a sample came from a known distribution

It does not make sense to me, based on a simple thought experiment that I do:

Lets say there are 2000 identical values. According to the distribution, this value has a 0.99 chance of being drawn. Rationally, it would make sense that this set of 2000 values came from this distribution. However, using the above answer the probability that the values came from the distribution is given by 0.99^2000 which is 1.86 * 10^-9

So, how do I calculate the probability that a set of values came from a distribution?

Personal attempts: I tried adjusting with a number of permutations, however the answer stopped making sense in the reverse thought experiment. Having a set of 2000 identical values, each of which have a probability of 0.1 of being drawn, the probability of the set coming from the distribution is then 0.1^2000 * 2000! which is 3.31 * 10 ^ 3735. So this also doesn't make sense.

I think what I'm actually looking for is a z-test or a t-test, but generalized to any distribution. Thus the approach would be to find the mean of the sample and compare it to the mean of the distribution.

The difficulty is that depending on the size of the sample, the probability of the sample mean being a D distance away from the distribution mean changes.

How to calculate this probability becomes the question. I do not see any other way but to use Markov Chain approach, and use computation repetitively create random samples of size S using the known distribution and create a distribution of distance D for sample size S. Then this distribution can be used to calculate the probability of the sample mean being different to the mean original distribution/model.

• Are we restricting consideration to discrete distributions here? – Glen_b Oct 13 '19 at 0:38
• We could. I am doing some computations, so it is easiest (on the brain at least) to make them descrete. – Anton Oct 14 '19 at 11:41
• Let's examine your hypothetical a little more closely. Suppose the underlying distribution were Normal. Generate independently 2000 values from it. The probability of that sample is zero. The point is that you are confusing two probabilities: the chance of obtaining a sample from a distribution versus the chance a distribution was the source of a sample. The latter cannot be computed until you (a) stipulate all the possible source distributions and (b) provide a probability distribution for them (a "distribution of distributions," aka a prior distribution). – whuber Oct 14 '19 at 16:20

Think like this: having observed 2000 outcomes all equal to $$x$$, you are trying to figure out if these observations came from, say, $$X$$ where $$Pr(X=x)=0.99$$. In absolute terms you get a very low probability of this happening. But compare to what? This distribution is still a better fit than, say, $$Y$$ where $$Pr(Y=x)=0.9$$ even as it is worse than, say, $$Z$$ where $$Pr(Z=x)=0.9999$$. So the result of your thought experiment is right. It's just that the probability has come lower than you expected.