This question is with regards to using a test data set to validate an imputed Cox model using R. With a non-imputed data set I would use val.surv() from rms, but I'm not sure how/if I can use it with my multiply imputed data set.

Further explanation: I created a predictive Cox PH model for 5-year RFS, and also used the mice package to multiply impute some missing data in the training data set. I then used fit.mult.impute() in Dr. Frank Harrell's excellent Hmisc package to obtain the pooled model. I have a data set that I would like to test the model on, but I am not sure how best to validate the pooled model.

Multiple imputation is a common procedure that many researchers utilize, so there must be a way that R users are validating their models with imputed data. I would like to know what functions/avenues are available for me to test this pooled model with my validation data set? Here is some sample code to work with:



#Set random data to NA 


#make a CPH for each imputation
for(i in seq(5)){

#Now there is a CPH model for mod_1, mod_2, mod_3, mod_4, and mod_5.


#Here is a test data set.
test_dat=data.frame(trt=replicate(500,NA), celltype=replicate(500,NA), time=replicate(500,NA), status=replicate(500,NA), karno=replicate(500,NA), diagtime=replicate(500,NA), age=replicate(500,NA), prior=replicate(500,NA))
for(i in seq(8)){

#Now there is a pooled model, "pooled_mod", and a test data set, "test_dat".

I'm looking forward to hearing about the R methods that can help in this situation.


1 Answer 1


I would suggest that you repeat every step that you would otherwise do on the non-imputed data for the first imputed data set. For example:

# fit the model on the multiply imputed data
fit <- with(impvet, cph(survmod ~ celltype + karno, x = T, y = T))

# take out the first imputed data set
data <- complete(fit, 1)

# take out the first fitted cph model
mod <- fit$analyses[[1]]

# use your val.surv() steps here on the first imputed data set

Of course, you can also study the second, third, … dataset.

  • $\begingroup$ I'm not sure I understand what you mean. Are you saying that I should validate only the model that is based on one of the imputations? Surely there is a way that R users are validating their models pooled from multiple imputations? Or is this a circumstance where SAS trumps R? $\endgroup$
    – JJM
    Jan 10, 2013 at 18:59
  • $\begingroup$ JJM, I understand your confusion. Preferably you should do all m analyses, and pool the results. My interpretation of your question was how to apply val.surv() in the context of multiple imputation. All I'm saying is: Apply it to the m imputed data sets, and not to "the pooled model". $\endgroup$ Jan 10, 2013 at 19:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.