Can I correct for randomly missing data where missingness is has a known relationship to the error term?

Question

Suppose I have a population of observations I want to model as being drawn from some distributional family, which I believe adequately represents the true distribution. My goal is to estimate the parameters of this distribution.

Suppose further that observations of my variable of interest are randomly missing, and the probability of missingness increases monotonically with the percentage by which the true value exceeds the value as estimated from the non-missing values, i.e. the error term in a semi-log or log-log regression of the variable of interest on the rest of the variables in the non-missing cases. In other words, for missing cases the value estimated by the regression is a lower bound of the true value. Missing observations appear as a visible blank – think of them as refusals to answer a survey questionnaire.

Finally, suppose that know that the probability that an observation will be missing is a known, increasing function of the (unobservable) error as described above, both as to form and parameters.

Can the visible observations from the population, together with the assumption of its distributional family, observations of missing values and the knowledge of the distribution of the probability of missingness, suffice to allow me to correctly estimate the parameters of the full distribution? If so, how? If not, why?

Answer 1

When you have missing data you need to (carefully) consider what is the missing data mechanism, i.e., how the probability of missingness depends on the observed an unobserved data. You wrote that the probability of missingingness depends on the true value of your variable of interest. This seems to be a missing not at random mechanism. To obtain unbiased parameter estimates in this case you will need to work with the joint distribution of the outcome and missingness processes. Based on the above description, the selection models seem to be appropriate, i.e., $$p(Y^o, M; \theta, \eta) = \int p(Y^o, Y^m; \theta) \, p(M \mid Y^o, Y^m; \eta) \, dY^m,$$ where $Y^o$ and $Y^m$ denote the observed and missing part of your variable of interest, $M$ is the missing data indicator, the first term in the integrand is your distribution of interest, parameterized by the vector $\theta$, and the second term specifies how the probability of missingness depends on $Y^o$ and $Y^m$ and is parameterized by $\eta$.