I have a few questions regarding on how diversity is defined since I've seen differing definitions in different papers.

In the paper "Measures of Diversity in Classifier Ensembles and their Relationship with Ensemble Accuracy" by Ludmila I. Kuncheva (2003) I saw this definition:

When classifiers output class labels, the classification error can be
decomposed into bias and variance terms (also called ‘spread’) (Bauer & Kohavi, 1999; Breiman, 1999; Kohavi & Wolpert, 1996) or into bias and spread > terms. In both cases the second term can be taken as the diversity of the ensemble.

However in other places I've seen that the diversity is based on how strongly correlated are the base learners. The less correlated they are the more diverse they are.

Or is there no difference since $Corr(X,Y) = \frac{Cov(X,Y)}{\sqrt{Var(X)Var(Y)}}$. The higher the variance the smaller the correlation and the larger the diversity will be(?)

As well, a high level definition I've heard too is that diversity measures the number of coincident errors committed by classifiers in an ensemble. Would this be a true statement still if the diversity was really the variance or correlation among classifiers?


Ensemble diversity, that is, the difference among the individual learners, is a fundamental issue in ensemble methods. diversity is crucial to ensemble performance. Generating diverse individual learners, however is not easy. The major obstacle lies in the fact that the individual learners are trained for the same task from the same training data, and thus they are usually highly correlated. individual learners should be accurate and diverse. Combining only accurate learners is often worse than combining some accurate ones together with some relatively weak ones, since complementarity is more important than pure accuracy. the success of ensemble learning lies in achieving a good tradeoff between the individual performance and diversity.

Reference : Zhou Zhihua (2012). Ensemble Methods: Foundations and Algorithms. Chapman and Hall/CRC. ISBN 978-1-439-83003-1.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.