Advice on handling missing data (high percentage of missingness for only one item)

Question

I'm working with data for a 30-item questionnaire that was administered at 24, 30, and 36 months of age. The data is largely complete, with the exception of age 36 months; at this age, we have one item (let's say item 11) that is missing like 90% of its data. This item is not missing anywhere near as much data at other ages. The missing data mechanism is MAR.

In a case like this, where 90% of the data is missing for the item, my understanding is that one would drop the item. The problem, however, is that this item is critical for producing an important summary score (90% of our summary scores are missing at age 36 months due to this issue). Because this item isn't missing at other ages, I do have an idea of what my expected distribution of values are. I've been imputing the item-level data using MICE with some success, but I've run into an issue.

Among satisfying other diagnostics, my understanding is the imputed data should have correlations (between the predictor and criterion variable) that match those of the original data. However, given than 90% of the summary scores (our predictor variable) are missing for age 36, I suspect that the predictor-criterion correlation in the original data is artificially high (especially compared to the other time points, where we have complete data). Although it doesn't match the stupidly high correlation from the original data, the correlation pooled from the imputed datasets seems much more reasonable and is more in alignment with the other time points, as well as the literature.

I'm not sure where to go from here. My PI tells me that the correlations from the imputed data should match those of the original data. However, this is going to be impossible, especially because the correlation we're attempting to "match" (which is way too high) is based off a summary score that is 90% missing. Would appreciate any advice on where to go from here.

Answer 1

The simplest way to deal with this is to recognize that the correlation coefficient of 0.50 estimated from 49 observations (as described in comments) is quite compatible with the value of 0.39 that you estimate from the imputed data.

The Fisher transformation is a standard way to estimate confidence intervals for correlation coefficients. The hyperbolic arctangent of the correlation coefficient $\rho$ is distributed approximately normally with standard error (SE) of $1/\sqrt{(n-3)}$, where $n$ is the sample size.

For a correlation coefficient estimate of 0.5 based on 49 observations, the 95% confidence interval (at $\pm 1.96$ SE) after back-transforming to the correlation scale is:

SE <- 1/sqrt(49-3)
tanh(atanh(0.5) + 1.96*SE)
# [1] 0.6849035
tanh(atanh(0.5) - 1.96*SE)
# [1] 0.2545947

The value of 0.39 estimated from the imputations is well within the 95% confidence interval (0.25, 0.68) of the estimate of 0.50 from the 49 cases. It's less than 1 SE away.

(atanh(0.5)-atanh(0.39))/SE
# [1] 0.9326118

The match seems quite good. Why complicate things further?