1
$\begingroup$

There are many posts about post hoc testing but I did not find an answer to my question.

my model:

 mod<-lmer(T ~ A*B + C + (1|D), REML=TRUE, data=dat) 

A,B,C are categorical with 2, 4 and 2 levels respectively.

I want to check the effect of the variable A on T (I use the package lsmeans but any other suggestion is welcome):

lsmeans(mod, pairwise~A)

I receive the warning:

NOTE: Results may be misleading due to involvement in interactions *

How can I evaluate the effect of A and A only considering the interaction?

NB: I know I can use lsmeans(mod, pairwise ~ A:B, adjust = "tukey") but then I obtain the effect of A for each level of B.

*I also carefully checked the documentation of the package lsmeans and there is only one example with interactions. However it turns out that the interaction did not influence the results and so how to include it in the results is not discussed.

$\endgroup$

1 Answer 1

4
+50
$\begingroup$

Presumably, the model includes an interaction between A and B because those factors actually do interact. That means that the effect of A is different at different levels of B, and vice versa. Therefore, wanting to "evaluate the effect of A and A only" can't be done; B has to be included in the picture.

First, as I often advise, statistics isn't about asterisks and P values, it's about understanding what is going on in the data. I suggest visualizing the effects first of all. For example, try

lsmip(mod, B ~ A)

which will produce an interaction plot showing how A changes for each B level. This plot may show widely varying effects. But it is even possible that the subjective effects of A are more or less the same across B, regardless of statistical significance of the interaction. If the latter is the case, it may make sense to do an overall comparison of A, averaged over B -- a result you already have (accompanied by the warning message shown).

But if the effects are not more or less the same, you should test each of those A effects separately at levels of B, as follows

lsmeans(mod, pairwise ~ A | B)
$\endgroup$
2
  • $\begingroup$ I particularly appreciated "statistics isn't about asterisks and P values", I would like to write it to some reviewer...Do you have any suggestion on how to check the strength of the interaction? $\endgroup$
    – have fun
    Sep 20, 2017 at 17:39
  • $\begingroup$ My suggestion is the same as for a lot of other things. Judge it both practically (does the lsmip plot ring any alarm bells, in terms of practical significance?) and statistically ($F$ test of interaction from ANOVA). You can get statistical significance for results that have no practical importance; and you can fail to have statistical significance for results that do appear to be practically important, if you have insufficient data to know it with any confidence. $\endgroup$
    – Russ Lenth
    Sep 21, 2017 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.