Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I feel like I've seen this topic discussed here before, but I wasn't able to find anything specific. Then again, I'm also not really sure what to search for.

I have a one dimensional set of ordered data. I hypothesize that all of the points in the set are drawn from the same distribution.

How can I test this hypothesis? Is it reasonable to test against a general alternative of "the observations in this data set are drawn from two different distributions"?

Ideally, I would like to identify which points come from the "other" distribution. Since my data is ordered, could I get away with identifying a cut point, after somehow testing whether it's "valid" to cut the data?

Edit: as per Glen_b's answer, I would be interested in strictly positive, unimodal distributions. I'd also be interested in the special case of assuming a distribution and then testing for different parameters.

share|improve this question
    
What do you mean by "same distribution"? Are observations of Gamma considered as come from the same distribution, or it is considered as the sum of exponential distributions? – Matemattica Jan 22 at 6:42
    
+1 this is a really good question for you to be asking yourself. – Mehrdad Jan 22 at 10:49
    
@Metallica as long as each observation is an exponential sum, I would say they are from the same distribution – ssdecontrol Jan 22 at 13:02
    
@Mehrdad I don't have formal statistics training beyond my undergrad degree and a few miscellaneous classes in my masters. If you look at my answer history, it's clear that I know a lot about linear regression and not a lot about anything else 🤐 – ssdecontrol Jan 22 at 13:07
2  
One possible way of approaching this question is to consider a finite mixture of e.g. some class of distributions and to see whether you need more than 1 mixture component to describe your data well. However, the question is whether there is a class of distributions that is sufficiently flexible to describe your "null hypothesis" by a single mixture component (e.g. if you use a finite mixture of gamma distributions these may not be flexible in terms of skewdness or tail behavior depending on what you are trying to do), while containing the potential alternative as a multi-component mixture. – Björn Jan 22 at 14:15
up vote 26 down vote accepted

Imagine two scenarios:

  1. the data points were all drawn from the same distribution -- one that was uniform on (16,36)

  2. the data points were drawn from a 50-50 mix of two populations:

    a. population A, which is shaped like this:

enter image description here

b. population B, shaped like this:

enter image description here

... such that the mixture of the two looks exactly like the case in 1.

How could they be told apart?

Whatever shapes you choose for two populations, there's always going to be a single population distribution that has the same shape. This argument clearly demonstrates that for the general case you simply can't do it. There's no possible way to differentiate.

If you introduce information about the populations (assumptions, effectively) then there may often be ways to proceed*, but the general case is dead.

* e.g. if you assume that populations are unimodal and have sufficiently different means you can get somewhere

share|improve this answer
1  
Thanks, great counterexample. So it comes down to appropriately restricting the alternative hypothesis, correct? – ssdecontrol Jan 22 at 13:03
    
@ssdecontrol yes, in essence; if (given the assumptions) the alternative is distinguishable from the null, you have some hope of a test with power higher than your significance level. – Glen_b Jan 23 at 1:35

You obviously need to have some theory to talk about distribution(s) and state hypotheses to test. Something that groups subjects in one or more groups and something that makes measurements to lay apart.

How can you get there? I see three options:

  • If you already know that from your subject matter, then you just need to translate it into the language of statistical hypothesis
  • Plot the charts and recognize patterns to become hypotheses to test
  • Come up with a list of distributions you could fit and do a mathematical experiment. Probabilistic programming is the keyword here

The exercise would then let you conclude that there are one or more groups represented in your sample or just one. Or no group at all.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.