The simple average is commonly used to combine the predictions of different predictive models. Apart form the simple average, what are the other methods that can be used for combining the predictive models to get more accurate predictions?


There are a few different ways. Here are examples:

  • Voting

If it's a classification problem, the models can vote on the outcome, establishing the class most models have selected. The voting procedure can be a straight one vote per model, or weighted, for example by confidence.

  • Super learner

The outputs of previously generated features are used as features in a new model - the Super Learner - which finds the best combination of those models. The recent R package SuperLearner is an implementation of this approach.

  • Averaging variations

Although you have mentioned averaging already, note that there are a number of different ways to weight the predictions in the averaging process. For example, you can assess the accuracy of the model, and then give more weight to the better performers. Or alternatively you can weight the prediction by confidence.

  • $\begingroup$ Super learner does apply different weights to the individual base models. your third example is repeating the second example. please expand on alternative weighting schemes besides super learning $\endgroup$
    – develarist
    Apr 5 '20 at 14:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.