Let's say we had 5 distributions: A,B,C,D,E.

An ANOVA test would tell us whether or not all of the means are equal, and thus a low p-value would mean at least one of the means is unlikely to be equal to the mean of the whole population of distributions. Is this a correct interpretation?

If so, I'm looking for something similar to an iterated ANOVA test, which tests all combinations of possible distributions to determine the likeliest number of distributions. So, if all 5 of the above had distinct means, it would return 5. If all 5 did not have distinct means and would not pass ANOVA, it would return 1. If all were distinct except D and E, then it would return 4.

Does such a statistical function exist with an implementation in R?


Your interpretation of a low p-value is a very common one, but it is not quite correct. The phrase "is unlikely to be equal" has no meaning in a Frequentist context. Either all of the population means are equal, or they are not. To the extent that something like a statement of probability can be made in this situation, the correct statement would be either 'the probability that all the population means are equal is $1$', or 'the probability that all the population means are equal is $0$'. Unfortunately, you don't know which of those statements is true, even after calculating a p-value. The meaning of the p-value is the probability of getting data (specifically, sample means) as far or further from equal, if the true population means are all equal.

With respect to the statistical function you are looking for, it sounds very much like running a one-way ANOVA, followed by Tukey's honestly significant difference testing procedure.

One last note: In the title, you ask about "distributions" in a way that connotes (to me, at least) any difference in the distributions (e.g., differing variances), but then refer more specifically to "means" only. If you were interested in the former, you might want to look into the Anderson-Darling test.

| cite | improve this answer | |
  • 1
    $\begingroup$ +1. The R implementation of Tukey's HSD is TukeyHSD. In using it, you will discover that this question typically has no unique or definite answer--there often is just a continuum of slight but insignificant differences among the groups, even though the extreme groups are significantly different. $\endgroup$ – whuber May 2 '13 at 15:44
  • $\begingroup$ @whuber is right: It is common to get a partial / incomplete picture of which groups are different. An example of this is discussed on CV here: how-to-interpret-a-post-hoc-tukeys-test. $\endgroup$ – gung - Reinstate Monica May 2 '13 at 15:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.