3
$\begingroup$

I have a training set where the inputs & outputs are all present, but I suspect that in the data where I want to do prediction, I will occasionally encounter scenarios where a small fraction of the input features are missing. Are there any machine learning methods that, once learning is complete, can provide reasonable prediction amidst missing inputs like this? If it matters, I'm looking for real-valued predictions (ideally multivariate, as I have 2 outputs to predict per input set).

$\endgroup$
  • $\begingroup$ You could try and use a Bayesian network for predicting some values, given a few other values. $\endgroup$ – Amulya Kishore Nov 29 '13 at 12:42
8
$\begingroup$

Substituting by the mean value is problematic and can lead to poor results. A principled way to tackle this problem is described in this paper. The idea is to formulate the problem in a probabilistic model which allows treating the missing components as hidden variables, and use the EM algorithm to estimate them. The paper also explains why is not recommendable to use the mean value.

If your model is a graphical model, then you can just integrate over the missing components. This gives you the most likely output compatible with the values of the observed components, averaged over all possible combinations of the missing values.

$\endgroup$
  • $\begingroup$ Nice paper reference...hadn't come across that one before. $\endgroup$ – jerad May 17 '13 at 20:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.