0
$\begingroup$

I started with this initial model:

m1 <- lmer(response ~ treatment + (1|subjectID), data = data)

However, the residuals of the model are heavy-tailed (presumably enough to violate the normality of residuals assumption?).

enter image description here

So, I tried a log transformation:

m2 <- lmer(log(response) ~ treatment + (1|subjectID), data = data)

Unfortunately, the residuals are virtually the same:

enter image description here

I also tried sqrt, cube root etc., with not much luck.

Finally, I tried a linear quantile mixed model, as it makes no assumptions about normality of residuals:

m3<- lqmm(response ~ treatment, random = ~1, group = subjectID, data = data)

Since m1 and m2 seem to violate the assumptions, I would have thought that lqmm (m3) is the most appropriate model. However, when comparing AICs, m2 (linear mixed model with log transformation) is HUGELY lower than m1 and m3:

AIC(m1, m2, m3)

enter image description here

I suppose my question is - would you choose the lqmm (m3) regardless of AIC, since m1 and m2 are arguably both invalid due to the non-normality of residuals? Or, would the considerably lower AIC of m2 give you 'pause' to suspect that it is the more appropriate model, despite deviation of residuals from normality?

For what it's worth, p values for one treatment group (compared to the reference group) are the same regardless of which model I use, but differ for two of the treatment groups compared to the reference group; m3 gives a significant p value for both of these comparisons, while m1 and m2 do not. In other words, the group 1 - group 4 comparison is always significant regardless of the model, but group 1 - group 2 and group 1 - group 3 comparisons are only significant in m3.

Apologies for any misunderstanding I may have and thank you for your help! I am not sure if I can provide the data due to confidentiality. Even if I provide only dummy variable names to ensure complete anonymity, I have to repeat this process many times over for different response variable and am therefore more interested in the conceptual reasoning for choosing one model over another, rather than an actual answer in this case.

Thank you again. :)

EDIT:

As requested, here are the residual vs fitted plots for m1 and m2:

m1:
enter image description here

m2:
enter image description here

Additionally, here are the QQ plots for the response variable split by group:

enter image description here

The '2 hour' group has more observations than the other three groups as all participants had a 2-hour observation, while each participant was randomised to one of the other groups (essentially within-subjects comparison when comparing 2 hours to all other groups and between-subjects when comparing the other three groups to each other, hence the random effect). There are 104 total observations across 55 participants.

From these plots, it looks like only the two-hour group is an issue - does that change the interpretation?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Welcome to CV! Can you share how many observations you have in the main data set? Let's say you have N observations. Before you confidently say that residuals are not normally distributed, you might want to plot multiple figures - each figure can have two sets of N numbers randomly sampled from a Gaussian distribution and plotted along each axis. These figures should allow you check if your residual plot is actually very different from what you see in the figures you plotted. $\endgroup$
    – medium-dimensional
    Commented Jul 24, 2023 at 8:06
  • 1
    $\begingroup$ Please also plot residuals vs fitted values to check if residuals have equal variance for m1 & m2. $\endgroup$
    – medium-dimensional
    Commented Jul 24, 2023 at 8:17
  • 1
    $\begingroup$ Thank you kindly for your responses - I have updated the post. @medium-dimensional, apologies if I misunderstood. $\endgroup$
    – Jade
    Commented Jul 24, 2023 at 8:32
  • $\begingroup$ I am also a learner ;) Please don't be worried to ask for clarification if you don't understand anything in the future on CV. Please check this GLMM FAQ and these GLMM worked examples - they might be helpful in answering your question. $\endgroup$
    – medium-dimensional
    Commented Jul 24, 2023 at 11:04
  • $\begingroup$ Please also check this post: when to check model assumptions. $\endgroup$
    – medium-dimensional
    Commented Jul 24, 2023 at 11:20

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.