1
$\begingroup$

I have a question regarding a post hoc test following an Anova.

I ran an Anova (response variable~Factor1xFactor2xFactor3)and found that I have 2 significant main effects, but no significant interactions :

Factor1 *** Factor2 *** Factor3 n.s. Factor1:Factor2 n.s. Factor1:Factor3 n.s. Factor2:Factor3 n.s. Factor1:Factor2:Factor3 n.s.

Factor1, Factor2 and Factor 3 all have 2 levels

I wondered now how to proceed from here. Would I simply do a post hoc test on the two main effects without an interaction term (response variable~Factor1 + Factor2)? Eventually, I'd like to have a barplot where letters above each bar indicate statistical differences, but if I want to do that, I would have to do all possible comparisons in an e.g. Tukey HSD test, which I don't do if I only test the two significant main effects without the interaction. Is that correct? Is it appropriate to run the post hoc test on the full model despite not having significant interactions? Can I infer from the Anova that certain comparisons are not significant without doing comparisons via a post hoc test?

Sorry for asking a question that is probably very basic to many of you but it got me really confused. All helpful answers will be greatly appreciated.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Re: "Is it appropriate to run the post hoc test on the full model despite not having significant interactions?"

Not generally. You would need a justification to retain a non-significant variable in the model you are doing Tukey post-hoc test on.

Run ANOVA with your two significant independent variables, without interaction ("Factor1 + Factor2" not "Factor 1 * Factor2). Run the Tukey post-hoc test on THAT result. You know Factor1 and Factor2 are significant, but you don't know which LEVELS of those variables are significant until you run the post-hoc test.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Hi Dale70, Thanks so much for your reply! This is exactly what I had in mind and what seemed logical to me. However, could I then still use a letter design to indicate signficant differences, given that I am not actually comparing all possible combinations with each other? $\endgroup$
    – Deschain
    Commented Aug 31, 2020 at 17:25
  • $\begingroup$ You can do an f-test to determine the overall significance of adding a variable to a model. Maybe you can alter your design to use that. $\endgroup$
    – Davis70
    Commented Aug 31, 2020 at 17:50
  • $\begingroup$ If a factor with only two levels is significant, what kind of post hoc test would you propose for that factor? $\endgroup$
    – BruceET
    Commented Aug 31, 2020 at 18:46
  • $\begingroup$ Are you asking me? One independent variable with two levels? You don't need a post-hoc for that. $\endgroup$
    – Davis70
    Commented Aug 31, 2020 at 18:58
  • $\begingroup$ I think I will just report Anova results then. In my specific case it only makes sense to make all comparisons or no specific comparisons. Just for my understanding, if it would happen to be a significant effect of factor 1 , significant effect of factor 2 and a significant interaction between factor 1 and 3, I would run a post hoc test like this: response~factor1 + factor 2 + factor1 x factor3 ? (purely hypothetical) $\endgroup$
    – Deschain
    Commented Sep 1, 2020 at 14:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.