As a simple example, say you want to predict the house price. You have 5 features. You build 5 models, each trained with one feature.
(price, sqft) (price, num_bedrooms) (price, lot_size) (price, num_bathrooms) (price, crime_rate)
For a test instance, you can just average the price predictions. That's one way to combine the models. Any other ways to combine models (perhaps a weighted average)?
If we average the results, how does will the results compare to just a similar model using all five features? Or how does building a bunch of simple models and combining them compare to building one complex model?
One thought is that perhaps this approach can be used when missing data is common. For example, just average the result of three models when sqft and bedrooms are missing for a test instance.
The other idea is to use the price outputs as features of a more complex model.