The structure of this question is as follows: at first, I provide the concept of ensemble learning, further I provide a list of pattern recognition tasks, then I give examples of ensemble learning algorithms and, finally, introduce my question. Those who don't need all the supplemental information might just look at the headlines and go straight to my question.
What is ensemble learning?
According to Wikipedia article:
In statistics and machine learning, ensemble methods use multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms alone. Unlike a statistical ensemble in statistical mechanics, which is usually infinite, a machine learning ensemble refers only to a concrete finite set of alternative models, but typically allows for much more flexible structure to exist among those alternatives.
Examples of pattern recognition tasks:
- Optical character recognition
- Barcode recognition
- License plate recognition
- Face detection
- Speech recognition
- Image recognition
- Document classification
Examples of ensemble learning algorithms:
The following ensemble learning algorithms used for PR tasks (according to Wiki):
Ensemble learning algorithms (supervised meta-algorithms for combining multiple learning algorithms together):
Boosting (a machine learning ensemble meta-algorithm for primarily reducing bias, and also variance in supervised learning, and a family of machine learning algorithms which convert weak learners to strong ones)
Bootstrap aggregating ("bagging") (a machine learning ensemble meta-algorithm designed to improve the stability and accuracy of machine learning algorithms used in statistical classification and regression).
Ensemble averaging (the process of creating multiple models and combining them to produce a desired output, as opposed to creating just one model. Frequently an ensemble of models performs better than any individual model, because the various errors of the models "average out.")
Mixture of experts, hierarchical mixture of experts
- Ensembles of neural networks (a set of neural network models taking a decision by averaging the results of individual models).
- Random forest (an ensemble learning method for classification, regression and other tasks, that operate by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes (classification) or mean prediction (regression) of the individual trees).
- AdaBoost (the output of the other learning algorithms ('weak learners') is combined into a weighted sum that represents the final output of the boosted classifier).
- Methods that use one neural network to combine different classifiers
- Competence areas method
Which of the ensemble learning algorithms is considered to be state-of-the-art nowadays and is actually used in practice (for face detection, vehicle registration plates recognition, optical character recognition, etc.) by enterprises and organizations? Using ensemble learning algorithms is supposed to increase recognition accuracy and lead to a better computational efficiency. But, do matters stand this way in reality?
Which ensemble method, potentially, can show better classification accuracy and performance in the pattern recognition tasks? Perhaps, some of the methods are outdated now, or have shown to be ineffective. It is also possible that ensemble methods now tend to not be used anymore on the strength of some new algorithms. Those who have experience in the area or have sufficient knowledge in this field, can you help to clarify the matters?