Train a neural network model that works even if some input features are missing

Question

Background

I am designing a NN model for a multi-class classification problem. The model takes two sets of features, F1 and F2, for making a prediction. However, F2 might be missing in production, but the model should still be able to make a prediction based on F1 alone (even at a degraded accuracy).

For example, suppose we have a list of objects. Some has only Color (F1) while others also have Weight (F2). I hope the model can work either with F1 only or with both F1 and F2.

Color (F1)	Weight (F2)	Class
Green	110 grams	Apple
Red	null	Apple
Red	130 grams	Apple
Orange	null	Orange

Question

How should I design the model?

I can think about two approaches:

• Train 2 models: first, train a model M1 on F1 only. Then train a model M2 that incorporates M1 as a submodule and also using F2. Then during inference time, if the input only has F1, we will call M1; otherwise, call M2.
• Train 1 model that takes both F1 and F2. But when building training data, we remove F2 of existing samples to create synthetic incomplete inputs that we might encounter during prediction.

What would be your suggestion? Any reading materials you recommend? Thank you for your time!

Result

In the end I went for approach 1 since it has better accuracy.

Answer 1

Even if you synthetically remove one feature while training, you still need to provide a numerical value for that (i.e. imputation). Or, you can directly resort to another model as described in your first option. Typically, the first option is more expensive when you have loads of features and you don't know which one will be missing, so imputation is a more economic choice. However, in your case, it doesn't seem like so. It's not possible which would work better (also your imputation method matters) without trying them out in a validation setting.