Machine Learning Framework that accepts Inputs with missing data and Outputs predicted values for that missing data

Question

Imagine a dataset of with values for 10 features for 100,000 samples. Some feature values are missing at random from some samples.

I would like to use this incomplete data set to train a single model with the following characteristics:

1. As input, this model will accept data from 1 sample that is incomplete for at least one of any of the features.
2. As output, the model will predict scores (and provide measures of certainty in regards to those predictions) for all missing values.

Thanks in advance for your insights and ideas!

Answer 1

If you're willing to specify a model for how all this data arises, then fitting a Bayesian model to the dataset is an option. Then you can draw missing values from the posterior predictive distribution and do not just get an average prediction, but also the uncertainty around it (by looking at the posterior predictive distribution).

This is called "multiple imputation". For several simple scenarios (e.g. you assume all values - at least after suitable transformations - follow a joint multivariate normal distribution) there are standard functions in most software packages to do this. For more complicated examples, you may have to hand craft this (which can be challenging for truly complex cases).

Answer 2

There are probably many ways of doing this.

I would turn to a k-nearest neighbours approach: use the provided features to determine the expected nearest neighbours, then set the unknown features to the mean of that feature among the neighbours. A corresponding measure of uncertainty would be the associated variance.

Using the same idea, and forgetting about your request of a single model, you could go further by training a predictor based on the neighbours only, in order to be able to capture local variations (especially for points which need to be extrapolated).