1
$\begingroup$

I am working with a large dataset --300,000 records-- which has many missing values. So I used Imputation to fill the missing values and more specifically the Chained-Equations Multiple Imputation method. In particular, I used three R packages to get a better feeling of their relative performance: Mice, Mi and MissForest. The imputation functions return imputation object from which I can extract the datasets. I have 10 datasets from each imputation function. My question is how to use them to run a Classifier and then use the model to Predict probabilities for the examples in my dataset.

One approach I used was to split the imputed datasets into training and testing subsets, then train a separate model --using a Classifier-- on each training dataset (e.g. 10 training datasets -> 10 models), consequently make separate predictions (probabilities) using each model separately on the respective testing dataset and finally average the predicted probabilities to end up with a single estimate for each example.

I wonder however if my approach is theoretically sound and if there is some other preferred method to combine imputed datasets and come up with a single prediction (probability) for each example.

Your advice will be appreciated.

==============================================================================

EDIT

#

Edit

#

I plotted the probability densities separately for the Positive and Negative examples in each imputed dataset separately (10 PDFs x 2). Then I plotted the averaged values of Probabilities across the imputed datasets and it became clear that this is definitely a wrong way to proceed. The distinction between Positive and Negative labels became entirely blurred.

Here is the code:

ggplot() +
  geom_density(data = compl1_Test,  aes(x = predict1_avts, fill = as.factor(NonCreditCard)), alpha = 0.4)  + 
  # coord_cartesian(xlim=c(0, 100000)) + 
  geom_density(data = compl1_Test,  aes(x = predict1ts, color = as.factor(NonCreditCard)), alpha = 0.5, size = 1) +
  geom_density(data = compl2_Test,  aes(x = predict2ts, color = as.factor(NonCreditCard)), alpha = 0.5, size = 1) +
  geom_density(data = compl3_Test,  aes(x = predict3ts, color = as.factor(NonCreditCard)), alpha = 0.5, size = 1) +
  geom_density(data = compl4_Test,  aes(x = predict4ts, color = as.factor(NonCreditCard)), alpha = 0.5, size = 1) +
  geom_density(data = compl5_Test,  aes(x = predict5ts, color = as.factor(NonCreditCard)), alpha = 0.5, size = 1) +
  geom_density(data = compl6_Test,  aes(x = predict6ts, color = as.factor(NonCreditCard)), alpha = 0.5, size = 1) +
  geom_density(data = compl7_Test,  aes(x = predict7ts, color = as.factor(NonCreditCard)), alpha = 0.5, size = 1) +
  geom_density(data = compl8_Test,  aes(x = predict8ts, color = as.factor(NonCreditCard)), alpha = 0.5, size = 1) +
  geom_density(data = compl9_Test,  aes(x = predict9ts, color = as.factor(NonCreditCard)), alpha = 0.5, size = 1) +
  geom_density(data = compl10_Test,  aes(x = predict10ts, color = as.factor(NonCreditCard)), alpha = 0.5, size = 1) 

And this is the graph: enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.