I have a large dataset with large amounts of missing data.

My data involves particular cognitive tests and I would like to see how they are related to academic attainment controlling for SES and IQ.

I would also like to impute missing values using multiple imputation.

I have been looking at the relationship between the amount of missing data each participant has, and both how well they do on the cognitive tests and in academic achievement. I have found that although there is no relationship between how much missing data one has and how well they do on cognitive tests, there is a relationship between how much missing data they have and how well they do at school, their SES and IQ. This could be for many reasons but I assume this means my data is not missing at random?

I was wondering whether if I ran multiple imputation on this dataset and included the correlated variables in the imputation model whether this would account for the relationship? Or if not, how I might be able to deal with this situation.

Many thanks!


The terminology might be getting in the way here. If the data that you have explain the probability that other data are missing, then your data might be "missing at random" (MAR) in the technical sense, even if they are not "missing completely at random" (MCAR). In that case multiple imputation is a reasonable way to proceed.

As Gelman and Hill put it in a chapter on missing data:

A more general assumption, missing at random, is that the probability a variable is missing depends only on available information. Thus, if sex, race, education, and age are recorded for all the people in the survey, then “earnings” is missing at random if the probability of nonresponse to this question depends only on these other, fully recorded variables.

Data are not "missing at random" if missingness depends either on unobserved predictors or on the values of the missing data points themselves. Unfortunately, there is no statistical way to document MAR status. If data are missing not at random (MNAR) there are ways to proceed, but you have to model the missingness mechanism. That might require some specific expertise and experience.

  • $\begingroup$ Great, thank you Ed. You're right, I think I'm getting MNAR and MAR confused. So if missing data can be explained by variables I have (ie. SES, IQ), would adding these variables in the imputation model be sufficient for this relationship to be accounted for? $\endgroup$ – GDon Sep 14 '16 at 15:59
  • $\begingroup$ Including all variables that might be related to the values of the missing data in your multiple imputation is about the best you can do. I'm aware, however, of no way to show that it truly is "sufficient," as missingness might also be related to unobserved predictors. That a general problem with imputation of missing data. Tests of the sensitivity of your results to your underlying assumptions are always wise if you need to convince yourself or someone else of the reliability of your work. $\endgroup$ – EdM Sep 14 '16 at 16:25
  • $\begingroup$ I have just come across another problem! I have missing data in my covariates and my outcome measures and they are all correlated with each other. Do I put these in the same imputation and impute together or should i impute separately? $\endgroup$ – GDon Sep 28 '16 at 14:25
  • $\begingroup$ @GDon the usual recommendation, as I understand it, is to do all imputations together, drawing on as much information as possible to get a reasonable set of imputations for all missing data. You want to take advantage of correlations among variables to get the imputations. Sometimes outcomes will provide information on covariates, other times it goes the other way. I don't, however, have direct experience with imputing outcome measures, just covariate values. As I understand it, that distinction shouldn't matter if the data are MAR. $\endgroup$ – EdM Sep 30 '16 at 17:43
  • $\begingroup$ Good comments by @EdM $\endgroup$ – Steffen Moritz Nov 11 '16 at 1:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.