Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I've got a dataset on agricultural trials. My response variable is a response ratio: log(treatment/control). I'm interested in what mediates the difference, so I'm running RE meta-regressions (unweighted, because is seems pretty clear that effect size is uncorrelated with variance of estimates).

Each study reports grain yield, biomass yield, or both. I can't impute grain yield from studies that report biomass yield alone, because not all of the plants studied were useful for grain (sugar cane is included, for instance). But each plant that produced grain also had biomass.

For missing covariates, I've been using iterative regression imputation (following Andrew Gelman's textbook chapter). It seems to give reasonable results, and the whole process is generally intuitive. Basically I predict missing values, and use those predicted values to predict missing values, and loop through each variable until each variable approximately converges (in distribution).

Is there any reason why I can't use the same process to impute missing outcome data? I can probably form a relatively informative imputation model for biomass response ratio given grain response ratio, crop type, and other covariates that I have. I'd then average the coefficients and VCV's, and add the MI correction as per standard practice.

But what do these coefficients measure when the outcomes themselves are imputed? Is the interpretation of the coefficients any different than standard MI for covariates? Thinking about it, I can't convince myself that this doesn't work, but I'm not really sure. Thoughts and suggestions for reading material are welcome.

share|improve this question
I haven't got the answer, but one question and two notes: 1) log of a ratio is, of course, the difference of logs. So your DV is equivalent to log(treatment) - log(control). 2) Which textbook of Gelman's were you looking at? –  Peter Flom Dec 19 '12 at 10:39
Yes, the DV is equivalent to log(treatment)-log(control). I'm basing the iterative regression imputation on the (nontechnical) chapter on missing data that Gelman has posted online: stat.columbia.edu/~gelman/arm/missing.pdf –  ACD Dec 19 '12 at 17:41
I have been told that imputing the outcome leads to Monte Carlo error. Will try to find a link later. Don't forget that you need to make sure to include the outcome in the imputation models for the covariates. –  D L Dahly Dec 30 '12 at 22:38

2 Answers 2

up vote 3 down vote accepted

As you suspected, it is valid to use multiple imputation for the outcome measure. There are cases where this is useful, but it can also be risky. I consider the situation where all covariates are complete, and the outcome is incomplete.

If the imputation model is correct, we will obtain valid inferences on the parameter estimates from the imputed data. The inferences obtained from just the complete cases may actually be wrong if the missingness is related to the outcome after conditioning on the predictor, i.e. under MNAR. So imputation is useful if we know (or suspect) that the data are MNAR.

Under MAR, there are generally no benefits to impute the outcome, and for a low number of imputations the results may even be somewhat more variable because of simulation error. There is an important exception to this. If we have access to an auxiliary complete variable that is not part of the model and that is highly correlated with the outcome, imputation can be considerably more efficient than complete case analysis, resulting in more precise estimates and shorter confidence intervals. A common scenario where this occurs is if we have a cheap outcome measure for everyone, and an expensive measure for a subset.

In many data sets, missing data also occur in the independent variables. In these cases, we need to impute the outcome variable since its imputed version is needed to impute the independent variables.

share|improve this answer
Thanks, this is consistent with my intuition, but could you perhaps share a link to a well-done published study that imputes dependent variables? One of the main reasons that I want to impute the outcome measures is to increase sample size (from about 250 to about 450), in order to facilitate semi-parametric tensor product interaction terms in GAM's that have very high df requirements (before they get penalized, lowering edf). MAR is reasonable in my case. –  ACD Jan 13 '13 at 6:37
It has been widely practiced for ANOVA to get balanced designs. See the introduction of RJA Little, Regression with missing X's, JASA 1992. I suppose that you know that increasing the sample size in this way does not help you to get more precise estimates. For the case of auxiliary variables, read the section on super-efficiency in DB Rubin, Multiple Imputation after 18+ Years, JASA 1996. –  Stef van Buuren Jan 13 '13 at 11:59
"Under MAR, there are generally no benefits to impute the outcome" - I have seen this mentioned before, but I don't have any reference for it - can you provide one please ? –  Robert Long Jan 14 '13 at 13:33
I think you can quote Little 1992 tandfonline.com/doi/abs/10.1080/01621459.1992.10476282 for that, but please note the exceptions. –  Stef van Buuren Jan 17 '13 at 17:04
@StefvanBuuren - helpful answer for the most part, but my understanding is that "if we know (or suspect) that the data are MNAR" then imputation cannot solve our problems any more than complete case analysis can. This seems to be fall in the "no free lunch" category. –  rolando2 May 26 at 22:21

Imputing outcome data is very common and leads to correct inference when accounting for the random error.

It sounds like what you're doing is single imputation, by imputing the missing values with a conditional mean under a complete case analysis. What you should be doing is multiple imputation which, for continuous covariates, accounts for the random error you would have observed had you retroactively measured these missing values. The EM algorithm works in a similar way by averaging over a range of possible observed outcomes.

Single imputation gives correct estimation of model parameters when there is no mean-variance relationship, but it gives standard error estimates which are biased toward zero, inflating type I error rates. This is because you've been "optimistic" about the extent of error you would have observed had you measured these factors.

Multiple imputation is a process of iteratively generating additive error for conditional mean imputation, so that through 7 or 8 simulated imputations, you can combine models and their errors to get correct estimates of model parameters and their standard errors. If you have jointly missing covariates and outcomes, then there is software in SAS, STATA, and R called multiple imputation via chained equations where "completed" datasets (datasets with imputed values which are treated as fixed and non-random) are generated, model parameters estimated from each complete dataset, and their parameter estimates and standard errors combined using a correct mathematical formation (details in the Van Buuren paper).

The slight difference between the process in MI and the process you described is that you haven't accounted for the fact that estimating the conditional distribution of the outcome using imputed data will depend on which order you impute certain factors. You should have estimated the conditional distribution of the missing covariates conditioning on the outcome in MI, otherwise you'll get biased parameter estimates.

share|improve this answer
Thanks. First off, I'm programming everything from scratch in R, not using MICE or MI. Second off, I am imputing with draws of a (modeled) predictive distribution, not just conditional expectations. Is that what you are talking about in the second paragraph? If not, I'd appreciate clarification. Also, which Royston paper are you referring to? For your last point -- are you saying anything more complicated than "you should put your dependent variable in the imputation model."? If so, I'd greatly appreciate clarification. –  ACD Jan 13 '13 at 18:08
Lastly -- I'm not doing single imputation. I'm fitting 30 models with filled in data and using the V_b = W + (1+1/m)B formula from Rubin. –  ACD Jan 13 '13 at 18:15
Royston paper was hyperlinked. I actually meant to link the Van Buuren one who implemented the program in R and includes computational details: doc.utwente.nl/78938 MICE/MI is a process. If you're imputing based on home-grown code, you ought to better elaborate on the details. Conditional means = predicted values if the model is correct (or approximately so, a necessary assumption). It is more complicated than "add the outcome", it's that you're imputing over several missing patterns (at least 3, missing covariate / outcome / jointly missing). –  AdamO Jan 13 '13 at 18:28
If you're singly imputing the predicted value 30 times, you should be getting the same results 30 times. How are you estimating the error? –  AdamO Jan 13 '13 at 18:29
Its a pretty simple algorithm -- say I observe a, b, c and d with some missingness. I fill in all four with random draws (with replacement) from observed values. Then I model imp = lm(a~b*+c*+d*) where * indicates filled in, and then x = predict(imp,se.fit=TRUE), y = rnorm(N,imp$fit,imp$se.fit). I then do a* = y, and then do imp = lm(b~a*+c*+d*), predict the same way, and so on. I loop through the whole set of variables 50 times. This is all from that Andrew Gelman textbook chapter that I linked above, and it is also why i don't get the same result each time. –  ACD Jan 13 '13 at 18:41

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.