I'm working on a model which has been fitted to longitudinal data (using mixed effects regression). I'm also investigating the effects of about 6 covariates on this model. Covariate A (continuous variable) is quite important in my investigation but the problem I have is that about 50% of these covariates are missing from my population. The question is at what percentage would you consider excluding a covariate completely? I'm a bit concerned that if I include only half of the covariate into my analysis, the result is likely to be unreliable or biased. I hope my question is clear enough but I would be happy to clarify anything. I've read here that $10\%$ is a good cut off. Is this correct?

  • $\begingroup$ Why are the data missing? Completely at random? Does the probability of an observation's being missing depend on what its value would have been? Is there some systematic reason why the data are missing? If you want to do predictions based on your model, will data be similarly missing for new observations? Depending on your answers to these questions, there may be very good ways to proceed or you might have a very difficult problem. $\endgroup$ – EdM Aug 5 '15 at 21:15
  • $\begingroup$ @EdM, thanks for your comment. The missing dataset is completely random-it was just not supplied with the original data. My model could be used in making predictions but I can't say with absolute certainty that future dataset will or will not contain missing values. $\endgroup$ – John_dydx Aug 5 '15 at 21:24

If you are confident that your data are missing completely at random (MCAR), or even just that the probability of a value's being missing is independent of what its value would have been ("missing at random," technically different from MCAR and a less restrictive requirement), then you should consider multiple imputation.

In this process, several sets of stochastic estimates (imputations) are made of what the missing values might have been, based on the relations within the data you have. You have some ability to determine which inter-variable relations will be used for imputation.

You then perform your analyses on each of these imputed data sets, and combine the analyses among the multiple imputations. This approach includes estimates of error arising from the imputation process, in addition to the errors always inherent in modeling. So the error estimates will include errors due to your missing half of the data on your important continuous covariate.

When the necessary assumptions are met, multiple imputation will perform better than limiting analyses to cases having complete data or relying on some arbitrary cutoff for deciding whether to include a variable at all. The web site linked above has links to tools for performing multiple imputation, including the mice package in R, and there is a multiple-imputation tag on this site that you can follow for further discussion.


Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.