In matlab, I've ran
[p,stats,~] = anova1(X)
with some samples, and it indicates a significant difference in (at least) 2 means from the samples. i.e. $p < \alpha$ for some value of $\alpha$.
However, running
multcompare(stats,'alpha',alpha)
doesn't indicate any significant difference between the data sets (i.e. there aren't any confidence intervals that don't contain 0).
Is this a contradiction, or is this possible, and why?
Many thanks.
I'm not familiar with the matlab functions you're using, but it looks like the first one does ANOVA and the second one does a series of pairwise comparisons between the means. The reason you get different results is because the two hypotheses being tested are different --
In ANOVA, you're just testing whether there is heterogeneity in the means of $k$ groups, i.e.
$$ H_0 : \mu_1 = \mu_2 = ... = \mu_k $$
When you do the pairwise comparisons you're testing a series of null hypotheses of the form
$$ H_0 : \mu_{i} = \mu_{j} $$
These two do not need to agree.
As an informal example, if all of the means were a "little" different from each other, you may reject the ANOVA null hypothesis but none of the pairwise differences are large enough to the individual null hypotheses. In another case, if all but one of the means were exactly the same, then you may not reject ANOVA null hypothesis, but there may be significant pairwise differences involving the one different group.
