2
$\begingroup$

I want to assess the impact of my intervention in a repeated-measures design. I have subject as a random intercept in order to account for the dependence of measurements within subjects:
Outcome ~ Condition*Time + (1|Subject), where Condition is treatment or placebo. I expected the effect of my intervention to increase over time, and I indeed see a significant Condition*Time interaction, which supports my hypothesis.

However, in reality, I guess that subjects will have variable effects of Time on Outcome. Indeed a model comparison confirms that the model Outcome ~ Condition*Time + (1+Time|Subject) fits the data significantly better, but my Condition*Time interaction is no longer significant in the presence of this random slope.

Now, I am wondering if the random slope may have masked the effect of my intervention, by falsely attributing the variation in the effect of Time to Subject, rather than to Condition. Is this possible?

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
5
$\begingroup$

The problem with the idea that the random slope in the mixed model is "falsely attributing the variation in the effect of Time to Subject, rather than to Condition" is that you are then assuming that Time:Condition interaction is ground truth--but this is the hypothesis you are trying to confirm. You could just as easily surmise that the former model is mistaking subject level variability as a Time/Condition interaction. Either way, you'd be cherry picking the model that gives the effects you want to see rather than interpreting possible effects from a model based in the design of your experiment and/or the nature of your hypothesis.

Neither model is true, but the significantly better fit (log-likelihood?, AIC?) suggests the mixed model might be better for analysis. Which model is best for testing your hypothesis and does the best job of accounting for sources of variability, error and/or bias? Are there other modes of analysis that might be better suited for your experiment based on its design? This is why it is often best to create an analysis plan before data collection and only deviate when necessary to deal with messy data (missing values leading to lack of balance, etc.)

Improve this answer
$\endgroup$
8
  • $\begingroup$ Thank you for that comment. I agree with your point on planning my analysis, and indeed the model without the random slope is pre-registered. The reason I'm asking the question is because the model with the random slope fits better according to all fit criteria provided by MATLAB: AIC, BIC and log-likelihood. However, in my opinion, the model that best checks my hypothesis is the pre-registered model, because I assume I've done a good job randomizing my sample. Therefore, I'd like to see any difference between conditions as a result of my intervention. Thanks again for your clarifications. $\endgroup$
    – galliwuzz
    Jul 30, 2018 at 10:42
  • $\begingroup$ I guess a different way of phrasing the question would be: Assume I had a very large effect of my treatment in reality. Could the random slope (1+Time|Subject) mask this effect? $\endgroup$
    – galliwuzz
    Jul 30, 2018 at 12:45
  • $\begingroup$ It wasn't so much the pre-registration that was key but the choice of model to best control for sources of variability and bias. For example, if your subjects are animals, subject level random slopes would control for slight differences in genetics that might result in variable response rates. $\endgroup$
    – deasmhumnha
    Jul 30, 2018 at 15:10
  • $\begingroup$ In general, if "large" (bit ambiguous) treatment effect is ground truth, then this should appear in both models; the random slopes should have little to no impact as evidenced by their small variance. So I'd say no. Any "masking" would instead mean your data set is too noisy to make conclusions and would suggest the need for more data or a repeat of the experiment with a different design. $\endgroup$
    – deasmhumnha
    Jul 30, 2018 at 15:12
  • 1
    $\begingroup$ @galliwuzz If you are using R, try the user guide for lmer. Otherwise, look at any source of mixed effects models. The exact fitting algorithm will depend on the implementation and the distribution family (linear mixed model vs generalized mixed model). But in general, the "decision" will be based on a greedy optimization of the (posterior) likelihood $\endgroup$
    – deasmhumnha
    Aug 24, 2018 at 4:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.