3
$\begingroup$

My 1-way ANOVA test is not significant. Is it possible that a Scheffé test as a post hoc test be significant?

If not, please say why not.

$\endgroup$
3
  • $\begingroup$ In general, the process of iteratively seeking tests that will deliver significant results is called "statistical fishing" and will greatly inflate your type 0.05 error rate. If the data truly aren't consistent with a trend, consider instead how you can communicate those results in a significant way to your collaborators. $\endgroup$
    – AdamO
    Oct 4, 2014 at 15:11
  • $\begingroup$ user48162 - No doubt @AdamO meant to type "Type I error rate" there. Is this for some exercise? $\endgroup$
    – Glen_b
    Oct 4, 2014 at 21:44
  • $\begingroup$ @Glen_b HAH. That's what happens when you switch from coffee to tea and check CV in the morning. Or maybe I'm onto a Fisherian epiphany regarding a fractal Hausdorff dimension of error rates. $\endgroup$
    – AdamO
    Oct 4, 2014 at 21:59

2 Answers 2

2
$\begingroup$

It is NOT possible. The ANOVA $F$ test is exactly equivalent to finding the most significant possible contrast and testing it against the Scheffé critical value. (Actually, that is how the Scheffé method is derived.) So if the $F$ is nonsignificant, so is every possible contrast.

$\endgroup$
1
$\begingroup$

@rvl's answer is correct for Scheffé's test, which is what the question was about. With Scheffé it is not possible for any comparison to be statistically significant when the overall ANOVA is not.

But for others reading this question and answers, note that it would be a mistake to generalize this answer to all multiple comparison tests. With the methods of Tukey or Dunnett, it is possible for a comparison between two group means to be statistically significant even though the overall ANOVA is not.

$\endgroup$
1
  • $\begingroup$ Why this happened for scheffe but not for happened for other multiple comparison tests? (just for scheffe?) $\endgroup$
    – Bernard
    Oct 5, 2014 at 8:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.