3
$\begingroup$

I have a paradigm where participants complete a reaction time task with three different stimulus 'Conditions'. I generate two contrasts for my mixed model. I produced the plots below to assess the normality of random effects. The random effect that is assessed here is (Condition|participant).

  1. Can someone explain what is meant by 'random effect quantiles' and 'standard normal quantiles' (the axes labels)?

  2. How is non-normality corrected? Transformations? Robust regression? Is this likely caused by participant outliers?

This is the plot produced for each fixed effect contrast 'Condition' in my linear mixed model. There are random intercepts for participant and by-participant random slopes for Condition

$\endgroup$
1
  • 1
    $\begingroup$ It's hard to know what exactly these are showing without your code; are they the residuals from a model? . I'm also not sure why there are error bars. And they don't even look that non-normal. $\endgroup$
    – Peter Flom
    Dec 5, 2023 at 12:53

1 Answer 1

3
$\begingroup$
  1. The quantiles on the x-axis represent what is often called 'theoretical quantiles' in QQ-plots; the distribution of quantiles in a standard normal distribution, in this case. The quantiles on the y-axis represent the observed quantiles, in this case the quantiles of the distribution of the predicted random effects.

    Each dot represents a level of the random effect. If the predicted values of random effects follow a normal distribution, they would lie on (or close to) the straight line. Deviations indicate:

    • Points below the straight line indicate the predicted values for those levels of the random effects are more concentrated than what would be expected for a normal distribution. I.e., some levels of the random effect have very similar or identical predicted values.

    • Points above the straight line indicate the predicted values of the random effect are more spread out than what would be expected for a normal distribution.

    In your left plot: The three left-most points indicate that for three levels of the random effect (probably three subjects?), the predicted value of the random intercept is close to -0.4. For a perfectly normal distribution, these values should be somewhat lower (more extreme / further from the mean). The two right-most points indicate that for two levels of the random effect, the predicted value of the random intercept is close to 0.75, while for a perfectly normal distribution, these value should be somewhat lower (less extreme / closer to the mean).

  2. I have no clear answer, I hope others will chime in. Some suggestions:

  • I would avoid transforming the response variable, because this will affect all components (both fixed and random) of your model.
  • These plots indicate departures from the assumed distribution. With departures, point estimates (e.g., estimated fixed effects) can often be trusted, but variances are affected, and thereby statistical tests (p-values) may become less trustworthy. Note that most of the random-effects predictions lie on the straight line, mimicking a normal distribution. The plots show a few outliers, for which some of the CIs also do not cover the straight line. I am not sure how worried one should be about this.
  • I would plot the predicted values of random effects (e.g., create a histogram), and perhaps try to identify which levels (subjects?) create these departures from normality. I am no big fan of throwing out outlying observations, but as a sensitivity analysis you could e.g., consider repeating the analyses without these subjects, to see if this affects conclusions.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.