1
$\begingroup$

I've run a GLMM with Gamma distribution and found a statistically significant effect of both of my independent variables. How do I run post hoc tests to examine these effects further?

The design is: I have two factors stimulation and temperature. The outcome is time until a response. I found a statistically significant effect for both factors. However, I further want to know direction of these effects. I am interested in further testing two things: if the response time is longer with stimulation; and if the response times are shorter with higher temperature.

I'm not sure if this is an appropriate application of posthoc tests, or should I just remark on the direction of effects in the first model?

Improve this question
$\endgroup$
2
  • $\begingroup$ So you've determined that both independent variables have a significant effect on your response variable. Can you be more specific about what additional tests you want to carry out? $\endgroup$
    – rw2
    Commented Apr 4, 2018 at 10:10
  • $\begingroup$ Apologies for the vagueness. So I have two factors i) stimulation, ii) temp. I found an effect of both. But how do I know test to show that i) the difference in relation to stimulation is that the behaviour is performed longer in the stimulation treatment versus the no stimulation treatment, and, ii) that the length of time performing the behaviour reduces with increasing temperature. I'm not sure how best to do this using posthoc tests, or is it sufficient to use the means to describe the differences? $\endgroup$
    – Gra
    Commented Apr 5, 2018 at 10:31

1 Answer 1

Reset to default
2
$\begingroup$

A theoretically sound alternative to testing, and post-hoc testing, is to just look at the 95% confidence intervals for effects and comment on their direction and magnitude. You can even plot the predicted distribution of wait-times for multiple exposure levels and comment on the their shapes and scales.

I would recommend this for a couple reasons: first is that the one-tailed tests were not preplanned. Rather, it seems that this hypothesis was sparked by your looking at the results. You cannot use data which generated a hypothesis to test a hypothesis; Second is that a one-sided statistical test will only tell you about the direction of effect, which is rarely as useful as the magnitude of effect. If the result is on the borderline statistically significant threshold, the one-tailed test will not be statistically significant and those results, altogether, are very puzzling. Conversely, showing that the 95% CI just barely misses the "null" threshold appropriately shows readers that, while statistically significant, the results are consistent with a mild to low range of effects and the study lacks power.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.