# “ all of these data points come from the same distribution.” How to test?

I feel like I've seen this topic discussed here before, but I wasn't able to find anything specific. Then again, I'm also not really sure what to search for.

I have a one dimensional set of ordered data. I hypothesize that all of the points in the set are drawn from the same distribution.

How can I test this hypothesis? Is it reasonable to test against a general alternative of "the observations in this data set are drawn from two different distributions"?

Ideally, I would like to identify which points come from the "other" distribution. Since my data is ordered, could I get away with identifying a cut point, after somehow testing whether it's "valid" to cut the data?

Edit: as per Glen_b's answer, I would be interested in strictly positive, unimodal distributions. I'd also be interested in the special case of assuming a distribution and then testing for different parameters.

• What do you mean by "same distribution"? Are observations of Gamma considered as come from the same distribution, or it is considered as the sum of exponential distributions? – Metariat Jan 22 '16 at 6:42
• +1 this is a really good question for you to be asking yourself. – Mehrdad Jan 22 '16 at 10:49
• @Metallica as long as each observation is an exponential sum, I would say they are from the same distribution – shadowtalker Jan 22 '16 at 13:02
• @Mehrdad I don't have formal statistics training beyond my undergrad degree and a few miscellaneous classes in my masters. If you look at my answer history, it's clear that I know a lot about linear regression and not a lot about anything else 🤐 – shadowtalker Jan 22 '16 at 13:07
• One possible way of approaching this question is to consider a finite mixture of e.g. some class of distributions and to see whether you need more than 1 mixture component to describe your data well. However, the question is whether there is a class of distributions that is sufficiently flexible to describe your "null hypothesis" by a single mixture component (e.g. if you use a finite mixture of gamma distributions these may not be flexible in terms of skewdness or tail behavior depending on what you are trying to do), while containing the potential alternative as a multi-component mixture. – Björn Jan 22 '16 at 14:15

Imagine two scenarios:

1. the data points were all drawn from the same distribution -- one that was uniform on (16,36)

2. the data points were drawn from a 50-50 mix of two populations:

a. population A, which is shaped like this: b. population B, shaped like this: ... such that the mixture of the two looks exactly like the case in 1.

How could they be told apart?

Whatever shapes you choose for two populations, there's always going to be a single population distribution that has the same shape. This argument clearly demonstrates that for the general case you simply can't do it. There's no possible way to differentiate.

If you introduce information about the populations (assumptions, effectively) then there may often be ways to proceed*, but the general case is dead.

* e.g. if you assume that populations are unimodal and have sufficiently different means you can get somewhere

[There restrictions that were added to the question are not sufficient to avoid a different version of the kind of problem I describe above -- we can still write a unimodal null on the positive half-line as a 50-50 mixture of two unimodal distributions on the positive half-line. Of course if you have a more specific null, this becomes much less of an issue. Alternatively it should still be possible to restrict the class of alternatives further until we were in a position to test against some mixture alternative. Or some additional restrictions might be applied to both null and alternative that would make them distinguishable.]

• Thanks, great counterexample. So it comes down to appropriately restricting the alternative hypothesis, correct? – shadowtalker Jan 22 '16 at 13:03
• @ssdecontrol yes, in essence; if (given the assumptions) the alternative is distinguishable from the null, you have some hope of a test with power higher than your significance level. – Glen_b Jan 23 '16 at 1:35

You obviously need to have some theory to talk about distribution(s) and state hypotheses to test. Something that groups subjects in one or more groups and something that makes measurements to lay apart.

How can you get there? I see three options:

• If you already know that from your subject matter, then you just need to translate it into the language of statistical hypothesis
• Plot the charts and recognize patterns to become hypotheses to test
• Come up with a list of distributions you could fit and do a mathematical experiment. Probabilistic programming is the keyword here

The exercise would then let you conclude that there are one or more groups represented in your sample or just one. Or no group at all.