3
$\begingroup$

I've been asked to provide standardized coefficients for a logistic mixed-effects model. The model contains several continuous predictors (which operate on similar scales) and 2 categorical predictors (one with 4 levels, one with six levels). The purpose of using the standardized coefficients would be to compare the impact of the categorical predictors to those of the continuous ones, but I'm not sure that standardized coefficients are the appropriate way to do so. For example, here it says

It is not sensible to standardize dummy regressors or interaction regressors.

My categorical variables are actually not dummy coded but factors, but I suppose the quote still applies.

The model is as follows: y ~ (categorical_1|SUBJECT) + categorical_1 + categorical_2 + continuous_1 + continuous_2 + continuous_3 + continuous_4 + categorical_1:categorical_2 + categorical_1:continuous_3

Note: y is a binomially distributed variable (measuring how often out of 6 trials a participant got the correct answer).

Is there a better way of comparing the impact of categorical and continuous predictors in this kind of model, or is it ok to use standardized coefficients here?

$\endgroup$
4
  • $\begingroup$ Are you talking about standardizing the variables prior to running the model, or standardizing the coefficients after? How are you treating the categorical variables if you are not using dummies ? Why is condition a randon slope but not a fixed effect also ? $\endgroup$ – Robert Long Sep 28 '20 at 4:18
  • $\begingroup$ 1) Yes, sorry, you are right, I'm talking about standardizing the coefficients. 2) Sorry, "condition" is actually "categorical_1" - I have edited the model code. So it is both a random slope and a fixed effect. 3) I'm computing the model in R, and my categorical variables are factor variables, and not numerical 'dummy variables' - this is what I meant. $\endgroup$ – Elinguist Sep 28 '20 at 9:32
  • $\begingroup$ 1) and 2), no problem, but 3) R will create dummy variables for the categorical variables behind the scenes. To see this, look at the output of model.matrix( ~ categorical_1 + categorical_2 + continuous_1 + continuous_2 + continuous_3 + continuous_4 + categorical_1:categorical_2 + categorical_1:continuous_3, mydata) $\endgroup$ – Robert Long Sep 28 '20 at 9:37
  • $\begingroup$ Thanks, good to know :)! $\endgroup$ – Elinguist Sep 28 '20 at 9:52
4
$\begingroup$

There seems be be a little confusion in the question. You appear to be talking about taking the output from your model and standardizing the coefficients, whereas the quote seems to be talking about standardizing the variables / regressors themselves.

It doesn't make much sense to standardize regression coefficients for variables that are categorical, or for interactions at all. Typically we divide by some measure of variability, often the standard deviation - but in the case of categorical variables, this is not appropriate. This obviously also applies to models including interactions with categorical variables. You say:

My categorical variables are actually not dummy coded but factors

The standard way to incorporate categorical variables into a regression model is with dummy variables, so the the output represents some kind of contrast - often between a reference level and the other levels.

Not only does it not make sense to standardize regression coefficients for variables that are categorical and interactions, but you are fitting a mixed effects model, so there is variation in the response that is due to the random effects and there is no consensus as to how to incporporate this, even in the case of global measures of fit such as $R^2$, so for individual measures of fit / effect size it should not be a surprise that this is just as, if not more, problematic.

It troubles me that reviewers ask for standardized coefficients for a model such as yours. I don't see anything wrong with interpreting the model without any kind of standardization.

It may be understandable that a reviewer may not know about issues with mixed models, so I would reply to them with a focus on issues with this for mixed model:

Unfortunately, due to the way that variance is partitioned in generalised linear mixed models, there is no agreed upon way to calculate standard effect sizes for individual model terms such as main effects or interactions (e.g., Rights & Sterba, 2919). We nevertheless decided to primarily employ mixed models in our analysis, because mixed models are vastly superior in controlling for Type I errors and non-independence than alternative approaches and consequently results from mixed models are more likely to generalize to new observations (e.g., Judd, Westfall, & Kenny, 2012). Whenever possible, we report unstandardized effect sizes which is in line with general recommendation of how to report effect sizes (e.g., Pek & Flora, 2018).

References:

Judd, C. M., Westfall, J., & Kenny, D. A. (2012). Treating stimuli as a ra ndom factor in social psychology: A new and comprehensive solution to a pervasive but largely ignored problem. Journal of Personality and Social Psychology, 103(1), 54–69. https://doi.org/10.1037/a0028347

Pek, J., & Flora, D. B. (2018). Reporting effect sizes in original psychological research: A discussion and tutorial. Psychological Methods, 23, 208–225. https://doi.org/10.1037/met0000126

Rights, J. D., & Sterba, S. K. (2019). Quantifying explained variance in multilevel models: An integrative framework for defining R-squared measures. Psychological methods, 24(3), 309. https://doi.org/10.1037/met0000184

where I have adapted this from here:
https://afex.singmann.science/forums/topic/compute-effect-sizes-for-mixed-objects

$\endgroup$
2
  • 1
    $\begingroup$ Wow, thank you so much for this detailed answer and for the reply with the references! It was actually not a reviewer who asked for this, but somebody who I personally asked for feedback on my empirical analysis. Yes, you are right, there was confusion in my question - I'm talking about standardizing coefficients, not the variables themselves. $\endgroup$ – Elinguist Sep 28 '20 at 9:28
  • 1
    $\begingroup$ You're welcome, and that's what I thought (about the standardization). $\endgroup$ – Robert Long Sep 28 '20 at 9:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.