# Dealing with missing data due to variable not being measured over initial period of a study

I was recently consulting a researcher in the following situation.

Context:

• data were collected over four years at around 50 participants per year (participants had a specific diagnosed clinical psychology disorder and were difficult to obtain in large numbers); participants were only measured once (i.e., it's not a longitudinal study)
• all participants had the same disorder
• the study involved participants completing a set of 10 psychological scales
• the 10 scales measured various things like symptoms, theorised precursors, and related psychopathology: the measures tended to intercorrelate around $r = .3$ to $.7$.
• in the first year one of the scales was not included
• the researcher wanted to run structural equation modelling on all 10 scales on the entire sample. Thus, there was an issue that around a quarter of the sample had missing data on one scale.

The researcher wanted to know:

• What is a good strategy for dealing with missing data like this? What tips, references to applied examples, or references to advice regarding best practice would you suggest?

I had a few thoughts, but I was keen to hear your suggestions.

• Does this study involve only one disorder (with low prevalence as I understand) or are there multiple diagnoses assessed by multiple indicators?
– chl
Sep 30, 2010 at 6:18
• @chl Just one disorder Sep 30, 2010 at 6:43
• Do the scales overlap to some extent (i.e. shared constructs across the questionnaires)?
– chl
Sep 30, 2010 at 6:45
• @chl they correlate but they are conceptually distinct; I've updated the question a little bit to reflect your two queries. Sep 30, 2010 at 6:49

I like the partial identification approach to missing data of Manski. The basic idea is to ask: given all possible values the missing data could have, what is the set of values that the estimated parameters could take? This set might be very large, in which case you could consider restricting the distribution of the missing data. Manski has a bunch of papers and a book on this topic. This short paper is a good overview.

Inference in partially identified models can be complicated and is an active area of research. This review (ungated pdf) is a good place to get started.

• Thanks for the link. What about imputation when data are not missing at random? It seems to me the problem lies in the fact that there is a complete block of measurements that is missing, and we cannot assume MAR or MCAR, nor are these data missing by design (in which case ML estimation yields unbiased parameters estimates).
– chl
Oct 2, 2010 at 19:10
• @chi MAR is exactly the sort of assumption that Manski's approach is meant to avoid. Perhaps a simple example will make it clearer. This is straight out of the paper I linked above. Suppose you want to estimate the average of some function of a variable y, E[g(y)]. Let z=1 for people with y observed, and z=0 otherwise. Also, suppose g is bounded between g0 and g1. Then you know that E[g(y)|z=1]P(z=1) + g0 P(z=0) < E[g(y)] < E[g(y)|z=1]P(z=1) + g1 P(z=0). E[g(y)|z=1] and P(z) can be estimated as the sample mean of the observed y, and the portion of the sample with y observed.
– paul
Oct 2, 2010 at 20:23
• Thanks for the explanation. I have no access to Manski's paper, but I will try to get it at work. (I have not vote left, so I'll +1 your response ASAP)
– chl
Oct 2, 2010 at 20:31
• @chi This ungated paper by Kline and Santos might also be useful. It focuses on quantile regression, but has references to similar papers about other models. econ.berkeley.edu/~pkline/papers/missing4.pdf
– paul
Oct 2, 2010 at 22:25