1
$\begingroup$

I understand that Fisher's LSD test is used as a post hoc analysis if the ANOVA test rejects the null hypothesis All the means $X_1, X_2,.. X_n$ taken from various samples are equal.

Also, given that if we reject null hypothesis via ANOVA, we can definitely find at least one pair of means $X_i \& X_j$ such that Fisher's LSD test is also violated. - This is what has been taught and proven in my stats course.

However, is it possible that we cannot reject the Null hypotheses via ANOVA and yet find of a pair $X_i , X_j$ such that Fisher's LSD will reject that both the samples have same mean. - I feel that this type of scenario is possible.

So, is Fisher's LSD ( for all the pairs) more stricter test compared to ANOVA for rejecting the null hypothesis?

edit: None of the post-hoc tests are guaranteed to find a significant pair even after rejecting main ANOVA. This is true with Fisher's test as described here, but also with a popular alternative like Turkey's test

$\endgroup$
  • 1
    $\begingroup$ "ANOVA" is not failing when it doesn't reject and it's not succeding when it does; similarly with post hoc testing. I have edited your title to remove the value-judgement. Further, the hypotheses are not about sample means they're about population quantities (that sample means differ is a matter of inspection). There are a number of other questions on site that deal with the issue of omnibus tests and post hoc tests not always corresponding.(one rejecting where the other would not). $\endgroup$ – Glen_b Sep 2 '17 at 21:42
  • 1
    $\begingroup$ @Glen_b I don't think it's a value judgment so much as saying a significance test "succeeds" is short for saying it successfully rejects its null hypothesis, and likewise for "fails". $\endgroup$ – Kodiologist Sep 3 '17 at 0:56
  • 1
    $\begingroup$ @Kodiologist I see just such phrasing quite often and it's almost always directly intended in the ordinary literal sense as a value judgement - and then functions as quite direct impetus for p-hacking. I think such phrasing should be resisted at every turn, in order not to further encourage the ingrained focus on significance as the criterion for success of a piece of research, which even today is widespread across many journals - including many which quite explicitly state in their editorial policies that this is not to be the case (e.g. when reviewers ignore that and editors allow them to) $\endgroup$ – Glen_b Sep 3 '17 at 1:16
  • $\begingroup$ @Glen_b I don't like significance testing any better than you do (possibly worse: I've yet to find a problem in social science where I think a significance test is helpful at all). I guess I just don't read much into "failed" as short for "failed to reject the null hypothesis". $\endgroup$ – Kodiologist Sep 3 '17 at 5:49
  • $\begingroup$ @Kodiologist: I just wanted your opinion on this concept; Given that if we reject null hypothesis via ANOVA, we can definitely find at least one pair of means Xi&Xj such that Fisher's LSD test is also violated. Do you think this statement always holds true? $\endgroup$ – kasa Sep 3 '17 at 5:53
4
$\begingroup$

So, is Fisher's LSD ( for all the pairs) more stricter test compared to ANOVA for rejecting the null hypothesis?

If I understand correctly, you're asking:

Is it true that for fixed $α$, if all the population means are equal and you test every pair with Fisher's LSD (with no requirement to conduct an ANOVA first), the probability that every pairwise null will be simultaneously rejected is strictly less than $α$?

The answer is yes, provided that $n > 2$ (and assuming, as usual, that the model is correct). To see this, note that the probabilities of rejecting $μ_1 = μ_2$ and $μ_2 = μ_3$ are each $α$. Call the former event $A$ and the latter $B$. By the definition of the model, $X_1$ is sampled independently of $X_2$ and $X_3$, so $B$ doesn't guarantee $A$, so $P(A|B) < 1$. Hence $P(A \wedge B) = P(A|B)P(B) = αP(A|B) < α$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.