Mixed model vs. imputation for questionnaire scale score?

Question

I would like to fit a statistical model where the dependent (response) variable is a validated scale score from a questionnaire. For each subject, this dependent variable is calculated from the values of a series of questions (items). If just one of the question answers is missing, this means that the scale score cannot be computed and is therefore currently coded as missing.

Im wondering how best to handle this. I have thought of three options so far. The first is to use multiple imputation on the component questions (not the scale score) so that we can achieve a complete record for each subject and subsequently compute the scale score. The second is to impute the scale score itself. The third is to run a mixed effects model without worrying about imputation since mixed models are known to be robust to missing data.

I would very much appreciate some guidance!

Answer 1

For multiple imputation, I think that your first two multiple imputation options will yield very similar results - assuming you are using the remaining complete responses for each record to impute. If you impute the answer to the question directly, you are sampling the variability in the unanswered question that is independent from variability in the answers to the other questions. If you impute the score, you are sampling the variability in the score that is independent of the other questions, which is pretty much the same thing. I'm not sure what kind of statistical model you're fitting, but there are Bayesian methods to deal with this kind of thing - sampling several posterior distributions with different draws of imputed data and combining to estimate posteriors for different parameters. A quick guide in R is here: https://cran.r-project.org/web/packages/brms/vignettes/brms_missings.html#fnref1

You could also just drop missing cases if they are scarce. One thing to consider is whether the missing data is random or not. If random, you're probably pretty safe, but if a particular grouping in your data is missing, this may bias your estimates. Of course, if this is the case and you use imputation you are resting on the assumption that the relationships elsewhere in your data hold in the missing section.

So either approach can work in my opinion, but be aware of the assumptions you are making in the method you choose. In many cases, you might try it both ways and find that you get similar results regardless.

Answer 2

Assuming that imputation is performed well, you are better off performing imputation. Thus, you will not have to drop either the samples with any missing values, or the features (ie question answers) that do not show up in all samples. (Having said that, there are certain predictive models, such as XGBoost, that are able to handle missing values. Thus, imputation is not strictly necessary.)

On the other hand, poorly-performed imputation (eg something naive, like mean imputation) could actually hurt performance. Personally, I would recommend UnmaskingTrees, a new method of which I am the author. UnmaskingTrees introduces new missing values and then trains XGBoost models to autoregressively impute them; then it incrementally applies these models to the actual missing values. Applied to a tabular imputation benchmark (developed by a different group), UnmaskingTrees had state-of-the-art performance as measured by downstream predictions (namely, F1 for classification and $R^2$ for regression).