Measures of goodness-of-fit using multiply imputed data in Zelig

Question

I am running a logistic regression model in R using multiply imputed data created using Amelia II, which I am then analyzing using Zelig. I would like to be able to report some measures of goodness-of-fit (e.g. likelihood ratio, pseudo R-squared, Hosmer-Lemeshow), however none are provided in the default Zelig output and I haven't been able to figure out a way to extract any from the zelig() object.

Do measures of goodness-of-fit need to be calculated differently when using multiply imputed datasets? Are there any R packages that are able to do this? I have looked into several packages that provide measures of goodness-of-fit, such as pscl, however they only work on glm objects, not MI objects created when using Amelia and Zelig.

Thanks in advance for your help!

Answer 1

This answer is concerned with the likelihood-ratio test only. I am unsure if there is a "general" paradigm of combining such measures.

For the LR-test you can look into the works of Meng & Rubin (1992) which you can find using Google Scholar. They describe how to combine the LR-statistic obtain from multiply imputed datasets. To calculate the resulting test statistic, you need access to the estimates of the two models that are to be compared for each dataset. Furthermore you need access to the likelihood function as the procedure asks you to evaluate it at user-defined values.

To my knowledge, implementations of this procedure are rare. You could have a look at SEM software such as lavaan in combination with the semTools package. The latter offers a function called runMI which imputes missing values, runs the analyses and gives you the combined model fit as per Meng & Rubin. If you already imputed your data using Amelia, you can save the resulting complete datasets as a list (of datasets) in which case runMI will only run the analysis and combination.

You will however, have to look into lavaan, which is designed for SEM and to mimic Mplus to some degree, and find out how to specify your model. I am not a 100% convinced by how the Meng & Rubin procedure is handled in semTools and Mplus, but is the only "general purpose" implementation of the combined LR-test that I am aware of.

Cheers!