Boosting is a strategy that uses such a heuristic. There's quite a body of literature about how and when it helps.
As for whether it is optimal for generalization on unknown test data: boosting is known to be more likely to overfit than e.g. an aggregation scheme without weighting. However, there's no free lunch, and no universal optimal heuristic here.
In particular, if you encounter a model with bad performance, there are totally opposite but equally sensible steps you could take:
- you already suggested that you should downweight models with low performance
- but low performance can also be an indicator of a reasonably well working algorithm that encountered difficult training and test data. And in that case you may want to upweight it.
Keep in mind that unless you have large sample size (which is typically not the situation where much thought is spent on ensemble models), the variance uncertainty due to finite test sample is often quite large. If this is non-negligible, boosting may lead to overfitting. The boosted ensemble may overestimate class separation.
Here's some literature:
- Bauer, E. & Kohavi, R. An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting and Variants, Machine Learning, 36, 105 - 139 (1999). than they actually are
- Zhang, M.; Xu, Q.; Daeyaert, F.; Lewi, P. & Massart, D. Application of boosting to classification problems in chemometrics, Analytica Chimica Acta, 544, 167–176 (2005). DOI: 10.1016/j.aca.2005.01.075
This paper deals with examples from my field (chemometrics). It does not compare weighted vs. unweighted aggregation but weighted aggregation vs. no aggregation. However, they give an example where boosting does not help at all.