I have been supporting the statistical assessment of a clinical study in which we have used a mixed models approach. For reference, the study looked at the error in subjects pointing at different height (or 'elevation angle') targets whilst blindfolded.
In writing this up I have tried to follow guidelines for reporting on mixed models whilst providing as straightforward a description of the process as possible. Below is an extract from the draft paper (irrelevant details removed for brevity):
In order to test the hypothesis that target angle affects JPS acuity, comparison was made between several linear mixed models fitted to the dataset using the R nlme package, with random effects to account for the multiple attempts per subject (Pinheiro et al., 2015). These within-subject attempts are assumed to be independent and free from correlation; this is based on [reasons given in paper].
Three models were estimated and compared, [...]: the first model explains the elevation error only as a random effect of the multiple within-subject attempts; the second explains elevation error as a fixed effect of the target angle and the random within-subject attempts; and the third is the same as the second but it also allows the variance of the error to differ between targets. All model fits were generated using the maximum-likelihood method, so that likelihood based methods could be used to compare models. Visual inspection of residual plots did not reveal any obvious deviations from homoscedasticity or normality. $P$ values were obtained by likelihood ratio tests between models.
Comparing the first two models, and visible from the plots in Figures 4 and 5, there is clear evidence that the elevation of the target affects subject elevation error. This is true in both abduction ($\chi^2 (2) = 244$, $p < 0.0001$), and in flexion ($\chi^2(2) = 299$, $p < 0.0001$).
Comparing the second and third model assesses the effect of target elevation on subject elevation error variance; evidence is found that target elevation does significantly affect elevation error variance in both abduction ($\chi^2 (2) = 33.0$, $p < 0.0001$), and in flexion ($\chi^2(2) = 9.4$, $p = 0.009$). In both cases the error variance is smallest at $90^\circ$and largest at $55^\circ$.
Unfortunately the paper has been rejected, based on a few different things, but generally around its structure and presentation, rather than content. Relating to the statistics the following comments came back from the reviewers:
Reviewer 1:
The statistical analysis is somewhat confusing. I'm not certain what was done even with the description. I would also suggest that the authors confirm all assumptions were met for the statistical analysis performed. The results should be expanded in the tables so that we can see means/standard deviations, etc.
Reviewer 2:
• Results are really ambiguous and unclear. I do not understand this "(x2(2) = 244 ..." Is this mean value? What is (2)? What does (244) mean? Is it angular position of the shoulder or something else?
• What is the statistical analysis test you used to detect significant difference? What do your numbers mean? What is the unit of measurement for those numbers? Too much ambiguity in your results.
I suspect that mixed-models are not a common approach in this area; however I'm unsure what extra information I can add that will clarify the details of the approach whilst remaining concise.
Is there a standard description or presentation of mixed model results that I am missing here? What details could be added to avoid similar comments from future reviewers (without embarking on a full description of mixed modelling from basic principles)?
