2
$\begingroup$

after researching on how to get estimated effect sizes from linear mixed effects models in R, I still do not understand how this can be properly done. Is there any function in R that does that? I am not lazy but I am working both with lmer and glmer on three types of data (reading times, reaction times, accuracy) and I really do not understand what there is to be done. People say that the estimated fixed effects coefficients are the effect size, or talk about standardization but I do not really understand what that implies really. I also do not really understand whether computing the effect size is different depending on coding scheme of the fixed effects (deviation vs treatment coding). Westfall and colleagues (2014) mention how the effect size could be calculated (the estimated slope coefficient for a given fixed effect / summed variances of all varying intercepts and slopes and residual variance) but I worry that the computing mode depends on the coding scheme, on whether people conduct lmer vs glmer analyses. On the other hand, how should one best report these results? It's all very confusing to me and I would definitely appreciate your help.

References:

Westfall, J., Kenny, D. A. and Judd, C. M. (2014). Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. Journal of Experimental Psychology: General 143(5): 2020–2045, DOI: https://doi.org/10.1037/xge0000014

Later edit: There seems to exist a function that computes this, which is implemented in the EMAtools package: lme.dscore. I am nevertheless interested in your input regarding the question that I posted and also with regard to prior experience with this function. Many thanks and happy Monday!

$\endgroup$
4
  • 1
    $\begingroup$ The term "effect size" can mean different things to different people. Why do you need to report an "effect size" instead of just reporting the regression coefficients and standard errors directly? If someone is requiring you to report an "effect size," which definition are you being expected to follow? $\endgroup$
    – EdM
    Oct 4 '21 at 14:52
  • $\begingroup$ My understanding is that Cohen's d is usually used to inform about effect sizes, but the textbook examples only report this value for simple linear models and I was wondering how it works for lmer. The fact that the term is so different for everybody makes me even more confused because I do not understand what the gold standard is. Thank you so much for taking the time to reply! I appreciate it. $\endgroup$ Oct 4 '21 at 15:11
  • $\begingroup$ Rather than "how do I get an effect size?", the first question to ask is "why do I need an effect size?" $\endgroup$
    – Russ Lenth
    Oct 6 '21 at 14:00
  • $\begingroup$ Dear Russ Lenth, thank you for taking the time to respond and comment on my question. I am not sure I am understanding your comment, but I gather that it invites one to do some critical thinking and not apply methods blindly - my willingness to report an effect size can be motivated. I am studying the effect of a specific type of information on language comprehension during reading. Since it is an effect that has not been studied much before, it makes sense to talk about effect size for future studies/ future power analyses. I hope my answer is in line with your question. $\endgroup$ Oct 7 '21 at 16:41
2
$\begingroup$

I personally find reports of "effect sizes" to be less compelling than seeing results in original scales. My hesitation to recommend "effect size" measures is enhanced by the extra difficulties in just what an "effect size" means in mixed models or generalized linear models.

The R emmeans package provides a way to calculate Cohen's d from many types of models when it makes sense, with an eff_size() function. See the pairwise comparisons section of the vignette for an introduction.

Nevertheless, the package author, Russ Lenth, discusses why presenting Cohen's d isn't always a good idea, in comments on this answer about the degrees of freedom (d.f) to specify to get an "effect size" for a mixed model. In particular:

In a simple lm() model, things are a lot more straightforward. In a mixed model, there is a lot more ambiguity. But to me, the d.f. to use is the smallest issue. What's bigger is deciding what you are even talking about when you compute an effect size. There is a lot of discussion on this, and from where I sit, the question is nearly unanswerable. I don't really believe in effect sizes. I provided the function because I thought [it] was important to account for uncertainty in the SD estimate if you insist on computing an effect size.

As Robert Long points out, Cohen's d doesn't make sense for generalized linear models. He recommends using odds ratios as a measure of "effect size" for logistic models.

See if you can present your results in the original scales as much as possible. That will remove ambiguity in the meaning of the term "effect size" and be most directly related to the things that you are measuring.

$\endgroup$
1
  • $\begingroup$ Dear EdM, I am really very thankful for your thorough answer! Thank you so so much for appending more answered questions. This is far more help than I expected to get! Thank you! $\endgroup$ Oct 4 '21 at 17:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.