I am designing a NN model for a multi-class classification problem. The model takes two sets of features, F1 and F2, for making a prediction. However, F2 might be missing in production, but the model should still be able to make a prediction based on F1 alone (even at a degraded accuracy).

For example, suppose we have a list of objects. Some has only Color (F1) while others also have Weight (F2). I hope the model can work either with F1 only or with both F1 and F2.

Color (F1) Weight (F2) Class
Green 110 grams Apple
Red null Apple
Red 130 grams Apple
Orange null Orange


How should I design the model?

I can think about two approaches:

  • Train 2 models: first, train a model M1 on F1 only. Then train a model M2 that incorporates M1 as a submodule and also using F2. Then during inference time, if the input only has F1, we will call M1; otherwise, call M2.
  • Train 1 model that takes both F1 and F2. But when building training data, we remove F2 of existing samples to create synthetic incomplete inputs that we might encounter during prediction.

What would be your suggestion? Any reading materials you recommend? Thank you for your time!


In the end I went for approach 1 since it has better accuracy.


1 Answer 1


Even if you synthetically remove one feature while training, you still need to provide a numerical value for that (i.e. imputation). Or, you can directly resort to another model as described in your first option. Typically, the first option is more expensive when you have loads of features and you don't know which one will be missing, so imputation is a more economic choice. However, in your case, it doesn't seem like so. It's not possible which would work better (also your imputation method matters) without trying them out in a validation setting.

  • $\begingroup$ Thank you. I am experimenting with training one model and imputing missing features. This will lead to a cleaner solution. Will see how it works and report it here. $\endgroup$
    – ZillGate
    Jul 10, 2022 at 19:34
  • $\begingroup$ There is this paper that makes neural network work with missing predictors in an elegant way! Check it out: arxiv.org/pdf/2206.01640.pdf $\endgroup$
    – Amin Shn
    Jul 11, 2022 at 12:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.