The simple average is commonly used to combine the predictions of different predictive models. Apart form the simple average, what are the other methods that can be used for combining the predictive models to get more accurate predictions?
There are a few different ways. Here are examples:
If it's a classification problem, the models can vote on the outcome, establishing the class most models have selected. The voting procedure can be a straight one vote per model, or weighted, for example by confidence.
- Super learner
The outputs of previously generated features are used as features in a new model - the Super Learner - which finds the best combination of those models. The recent R package SuperLearner is an implementation of this approach.
- Averaging variations
Although you have mentioned averaging already, note that there are a number of different ways to weight the predictions in the averaging process. For example, you can assess the accuracy of the model, and then give more weight to the better performers. Or alternatively you can weight the prediction by confidence.