I work with longitudinal data that tends to be “messy.” For example, we collect eye-tracking and physiological measures at multiple time points in young children. This causes data to be missing not only because of attrition (e.g., a child missing one time point), but also because a child may be moving too much or touching the equipment, creating too much noise in the data. Another example would be a child that does not have enough information in the recording session (e.g., not completing enough trials). Should these missing data be imputed just like the data missing because of attrition or should it be left missing?

Thanks in advance!

  • $\begingroup$ I don't understand why at the beginning you said there are a potential reason for missing value, and at the end for simplicity you assume missing values are at random? $\endgroup$ – Metariat May 12 '16 at 9:03
  • $\begingroup$ Sorry about that, @Matemattica. I just edited the question to reflect your comment. What would your recommendation would be to take into account the cause? Thank you very much! $\endgroup$ – san May 12 '16 at 16:53

Note here: throughout I refer to imputation and mean specifically multiple imputation. Any other form is biased and incorrect.

Imputation is actually a procedure that makes a lot of sense. I would be in the camp to recommend doing it always. However, it is often not performed because it is rarely the case that missingness is so prevalent that a complete case analysis fails to obtain significant findings whereas an imputed analysis would. The assumptions are easy to state, but there are no tests to assess them: so they must be taken with a grain of salt.

Imputation gives a power boost to your analyses. That boost is proportional to a few, somewhat complicated considerations. Imputation works best when many variables are missing in small proportions such that a complete case analysis might render 60-30% completeness, but each variable is perhaps only missing 10% of its values. The power boost is much less impressive when one important variable is missing in 70% of cases, because the uncertainty in estimates will yield highly varying imputed datasets.

Imputation usually doesn't introduce many additional assumptions in most analyses, and can be quite robust. In longitudinal studies, however, one must take care to assess data for informative missingness. This is not a statistical consideration in the sense that tests may be applied, or in which sensitivity analyses can be conducted: this is simply a practical consideration.

Informative missingness is when inclusion probability or loss-to-follow-up depends on unmeasured factors in the analysis. This is especially the case when dealing with high risk populations and high risk behavior, such as drug use, suicidality, or even education (children who dropout are likely to have dropped out because of poor performances). The effect is that of unbalanced design and non-representativeness. The effect is somewhat similar to unmeasured correlation in linear regression: it doesn't bias your analyses, it just gives incorrect inference. That can go either way, really. It's totally conceivable that a complete case analysis gives significant results and the imputed data (despite having more observations) is not significant--and the imputed inference is correct.

Imputation technically isn't necessary when using fully parametric methods such as Bayesian regression or maximum likelihood. This is somewhat different, but if you write out the full likelihood integrating over the missing values, you'll see that the full likelihood is proportional to the complete case likelihood, so they have the same maximum. In my thesis (unpublished) we did show that, despite this, you get power boosts from imputation. So there are reasons to still do this.

ref: statistical analysis with missing data, Little & Rubin 2002 2nd ed

  • $\begingroup$ Really nicely put (and succinct) answer! I'm going to save this, so that I can refer people to it. Thanks. $\endgroup$ – Jeremy Miles May 12 '16 at 17:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.