2 update with a link to NIPS paper edited Jul 17 '14 at 14:48 amoeba 64.6k1616 gold badges216216 silver badges273273 bronze badges This question is not restricted to LDA, but can be asked about any binary classifier that is used in a multi-class setting by making all pairwise comparisons. The question is how to combine all pairwise classifications into one final classification. The simplest approach is as follows. Each of the $$\frac{K(K-1)}{2}$$ pairwise classifiers results in a "winning" class (among the two considered). Count the number of wins for each of the classclasses (with the upper bound $$K-1$$), and assign the observation to the class with most wins. Note that this simple "voting" approach works even if your classifier does not return a probability of belonging to each of the two classes, but simply reports pairwise decisions. When each of the pairwise classifiers reports not only pairwise decisions, but also probability of belonging to each of the two classes, more sophisticated algorithms become possible. I cannot give an overview or an advice, but there is a massively popular 2004 paper (over 1k citations according to Google Scholar) that reviews exactly this question and offers some novel methods: Wu, T. F., Lin, C. J., & Weng, R. C. (2004). Probability estimates for multi-class classification by pairwise coupling. The Journal of Machine Learning Research, 5, 975-1005. I would guess, however, that in many real situations the simple voting method would already give reasonable results. Update: In the NIPS version of the same paper the authors report performance of several methods, including the "voting" one, on several real datasets with number of classes ranging from 6 to 26, see Table 1. The voting method seems to be very competitive in each case. On some datasets it even seems to outperform all other, much more sophisticated, methods. This question is not restricted to LDA, but can be asked about any binary classifier that is used in a multi-class setting by making all pairwise comparisons. The question is how to combine all pairwise classifications into one final classification. The simplest approach is as follows. Each of the $$\frac{K(K-1)}{2}$$ pairwise classifiers results in a "winning" class (among the two considered). Count the number of wins for each of the class (with the upper bound $$K-1$$), and assign the observation to the class with most wins. Note that this simple "voting" approach works even if your classifier does not return a probability of belonging to each of the two classes, but simply reports pairwise decisions. When each of the pairwise classifiers reports not only pairwise decisions, but also probability of belonging to each of the two classes, more sophisticated algorithms become possible. I cannot give an overview or an advice, but there is a massively popular 2004 paper (over 1k citations according to Google Scholar) that reviews exactly this question and offers some novel methods: Wu, T. F., Lin, C. J., & Weng, R. C. (2004). Probability estimates for multi-class classification by pairwise coupling. The Journal of Machine Learning Research, 5, 975-1005. I would guess, however, that in many real situations the simple voting method would already give reasonable results. This question is not restricted to LDA, but can be asked about any binary classifier that is used in a multi-class setting by making all pairwise comparisons. The question is how to combine all pairwise classifications into one final classification. The simplest approach is as follows. Each of the $$\frac{K(K-1)}{2}$$ pairwise classifiers results in a "winning" class (among the two considered). Count the number of wins for each of the classes (with the upper bound $$K-1$$), and assign the observation to the class with most wins. Note that this simple "voting" approach works even if your classifier does not return a probability of belonging to each of the two classes, but simply reports pairwise decisions. When each of the pairwise classifiers reports not only pairwise decisions, but also probability of belonging to each of the two classes, more sophisticated algorithms become possible. I cannot give an overview or an advice, but there is a massively popular 2004 paper (over 1k citations according to Google Scholar) that reviews exactly this question and offers some novel methods: Wu, T. F., Lin, C. J., & Weng, R. C. (2004). Probability estimates for multi-class classification by pairwise coupling. The Journal of Machine Learning Research, 5, 975-1005. I would guess, however, that in many real situations the simple voting method would already give reasonable results. Update: In the NIPS version of the same paper the authors report performance of several methods, including the "voting" one, on several real datasets with number of classes ranging from 6 to 26, see Table 1. The voting method seems to be very competitive in each case. On some datasets it even seems to outperform all other, much more sophisticated, methods. 1 answered Jul 16 '14 at 21:14 amoeba 64.6k1616 gold badges216216 silver badges273273 bronze badges This question is not restricted to LDA, but can be asked about any binary classifier that is used in a multi-class setting by making all pairwise comparisons. The question is how to combine all pairwise classifications into one final classification. The simplest approach is as follows. Each of the $$\frac{K(K-1)}{2}$$ pairwise classifiers results in a "winning" class (among the two considered). Count the number of wins for each of the class (with the upper bound $$K-1$$), and assign the observation to the class with most wins. Note that this simple "voting" approach works even if your classifier does not return a probability of belonging to each of the two classes, but simply reports pairwise decisions. When each of the pairwise classifiers reports not only pairwise decisions, but also probability of belonging to each of the two classes, more sophisticated algorithms become possible. I cannot give an overview or an advice, but there is a massively popular 2004 paper (over 1k citations according to Google Scholar) that reviews exactly this question and offers some novel methods: Wu, T. F., Lin, C. J., & Weng, R. C. (2004). Probability estimates for multi-class classification by pairwise coupling. The Journal of Machine Learning Research, 5, 975-1005. I would guess, however, that in many real situations the simple voting method would already give reasonable results.