My hypothesis states that "there would be significant differences among a, b, c and d group on the X measure." I used ANOVA to see the group differences which came out to be significant. After that I applied the Tukey test post-hoc to understand the comparisons further. However, I did not get significant group differences after Tukey. My supervisor says that the hypothesis is partially accepted. Can you explain how or if this statement is correct?
$\begingroup$
$\endgroup$
3
-
4$\begingroup$ I would not have a problem w/ claiming that there were differences among a, b, c, & d, under these circumstances. I would not use the phrase "partially accepted". $\endgroup$– gung - Reinstate MonicaCommented Feb 15, 2014 at 2:33
-
1$\begingroup$ This is what is bothering me - the use of the phrase partially accepted! $\endgroup$– Parul KathuriaCommented Feb 15, 2014 at 2:46
-
$\begingroup$ can you give example of partially accepted hypothesis with significance level $\endgroup$– user89994Commented Sep 21, 2015 at 9:04
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
9
Your ANOVA was significant, implying you either made a Type I error or the means are not all equal (in which case the null is false).
Since the chance of making a Type I error was (presumably) set fairly low, the second option becomes a relatively plausible explanation for the size of the test statistic.
In that sense, the research hypothesis you stated is indicated.
However, your multiple comparisons were unable to clearly identify any specific 'cause' of that difference - likely there are several small effects that are enough for yout to conclude there's a difference, even though none alone are large enough to 'stand out' by themselves for you to say "this pair of groups differ on X".
(Such a thing happens not infrequently, especially when samples size calculations are based on only just achieving a moderate power at some overall effect size. If the effect sizes are all a little smaller than that, you may be unlikely to find them.)
Edit: To address the specific phrasing of the research hypothesis being 'partially accepted' -
It depends on what you mean by "correct".
I would not use such a phrase - either accepting the alternative or 'partial' in reference to it. You rejected the null, and there was nothing partial about that.
I think the important thing is to convey exactly what null was rejected.
I'd also draw clear displays of means and (ANOVA-based) standard errors of the mean (likely along with the raw data on the same display) in order that the effect sizes relative to the uncertainty was clear to the readership.
I certainly have never used such phrasing and don't imagine I ever will, but that doesn't make it objectively wrong. What matters most is that the audience of such a phrase clearly understand the intended meaning.
-
$\begingroup$ I want to understand that is the phrase partially accepted correct to use? $\endgroup$ Commented Feb 15, 2014 at 2:47
-
$\begingroup$ I have now addressed that in the answer. $\endgroup$– Glen_bCommented Feb 15, 2014 at 2:53
-
$\begingroup$ My adviser is quite adamant on this. According to him the discussion is to be based on the Tukey results! And ANOVA should not be considered while discussing the results. And that is why the use of the phrase "partially accepted" $\endgroup$ Commented Feb 15, 2014 at 2:53
-
1$\begingroup$ Then why do the anova at all, if it is to be ignored? And if you work at your advisor's insistence irrespective of what we think, what point in asking us? We have no influence over your advisor, and even if there were some filtering back of information, I've learned from long, bitter, and at times quite agonising experience that one cannot teach an advisor better practice via the intermediary of their students. If I had your supervisor's ear I could explore the reasons for such insistence, but I cannot. If you must follow what they insist on, then better to go ahead and be resigned to it. $\endgroup$– Glen_bCommented Feb 15, 2014 at 3:08
-
$\begingroup$ I feel that my adviser was not correct on making such a statement! I wanted to confirm this and this was the only reason for me to post the question. $\endgroup$ Commented Feb 15, 2014 at 3:09
$\begingroup$
$\endgroup$
Tukey multiple comparisons control for multiplicity in the strong sense, meaning they control for every paired comparison simultaneously. Assuming there are no differences, then the probability of any Tukey comparison rejecting H0 is 5% (with 5% significance level). Therefore the Tukey test can be performed directly, without conducting the overall ANOVA F test first. On the other hand, the overall F test from ANOVA controls type I error in the weak sense, meaning it only controls for the overall hypothesis. Strong control is recommended over weak control, since most of the time we are interested in the specific paired hypotheses in practice, so you could adopt a practice of only doing the Tukey tests.
Rejecting the overall test implies rejection of the overall null that all means are equal. That is the only conclusion that can be made this this test. So you can claim there are differences, but you do not have enough evidence to say which specific ones are different without risking higher Type I error rate due to the multiplicity involved.
