I've run into this issue a few times now, with reviewers requesting more justification for the use of LMMs, traditional tests instead of or in addition to LMMs, and full tables of parameter estimates akin to what you would report with a regular linear model.
Right now my specific issue is a reviewer requesting "A table containing the main parameter estimates of the various models". I am thinking they want something like a traditional table one would report for a linear model (with t tests and p values), but in this case the analyses involve nested model comparisons and there are no t tests for each of the parameters included in each model, but rather a single test for the model comparison, which I do report in the paper. So I'm not sure what to do -- I want to satisfy the reviewer, but I don't necessarily want to include huge tables of information that are of little use to evaluating the results. Right now I simply report the beta, SE, chi-square and p value. I also make it clear what variables were included in each model. Any suggestions for how to proceed?
Here is what I am proposing to respond:
We believe the reviewer is asking for something akin to what would be reported in a traditional multiple regression analysis, with parameter estimates and their accompanying statistics and p values for each variable included in a given model. However, because linear mixed model analyses use nested models comparing reduced models to full models with one additional parameter, the only parameter that is tested is the one that is added in the full model (CITATION) As such, including a table would not support interpretation of the results in the way that it would in a more traditional analysis. Thus for each analysis, we report the betas for the tested parameter in each model comparison, along with the key statistics, in the body of the results section, as is recommended (CITATIONS).
Also, when asked for a justification for the use of LMMs in my particular case, this is what I'm proposing to respond:
We used linear mixed models because this analysis allowed us to account for variability due to trial type in our models (switch versus no-switch trials), while simultaneously accounting for the fact that trials were nested within subjects, and multiple responses from the same person are more similar than responses from other people. Accounting for both trial type and subject-level variance in reaction times was expected to reduce error in our models and increase our ability to detect any effect of task performance.
If you have any suggestions for how this could be improved, I'd appreciate it. Again, this audience is not statistically sophisticated, so adding tables and supplementary data is only likely to add to their confusion/skepticism.
Also, note that my motivation for using LMMs is different from what I've seen in papers (e.g., modeling multiple random effects simultaneously - in my case, there's only one random effect - participants, and trial type is a fixed effect), so I am not sure that citing some of the common papers is that helpful. It's possible that I've overlooked other ways to analyze this data so my justification for using LMMs is not apt.