4
$\begingroup$

In matlab, I've ran

[p,stats,~] = anova1(X)

with some samples, and it indicates a significant difference in (at least) 2 means from the samples. i.e. $p < \alpha$ for some value of $\alpha$.

However, running

multcompare(stats,'alpha',alpha)

doesn't indicate any significant difference between the data sets (i.e. there aren't any confidence intervals that don't contain 0).

Is this a contradiction, or is this possible, and why?

Many thanks.

$\endgroup$
1

1 Answer 1

4
$\begingroup$

I'm not familiar with the matlab functions you're using, but it looks like the first one does ANOVA and the second one does a series of pairwise comparisons between the means. The reason you get different results is because the two hypotheses being tested are different --

In ANOVA, you're just testing whether there is heterogeneity in the means of $k$ groups, i.e.

$$ H_0 : \mu_1 = \mu_2 = ... = \mu_k $$

When you do the pairwise comparisons you're testing a series of null hypotheses of the form

$$ H_0 : \mu_{i} = \mu_{j} $$

These two do not need to agree.

As an informal example, if all of the means were a "little" different from each other, you may reject the ANOVA null hypothesis but none of the pairwise differences are large enough to the individual null hypotheses. In another case, if all but one of the means were exactly the same, then you may not reject ANOVA null hypothesis, but there may be significant pairwise differences involving the one different group.

$\endgroup$
1
  • 1
    $\begingroup$ Ok, thanks. I think I assumed that if ANOVA rejects H0 then you could use multcompare (a multi-comparison test) to see which of the means are different, but I suppose its possible that you can't infer anything in general about multcompare rejecting H0 if ANOVA rejects H0 or vice versa. Thanks again. $\endgroup$
    – maliky0_o
    Mar 18, 2012 at 13:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.