In the scenario that I have a binary classification problem, and use a binary classifier to train and test my model, assuming everything else is constant, would using a multi-class classifier with 2 labels (one vs. all) produce the same results? (I know I should use the binary classifier, but I'm simply curious).
1 Answer
"Binary classification" is simply multi-class classification with 2 labels. However, several classification algorithms are designed specifically for the 2-class problem, where the response is modeled as the outcome of a Bernouilli trial.
Of course, this can be generalized to the n-dimensional setting by modeling as the outcome of a multinomial sample. For instance logistic regression can be generalized to multinomial logistic regression.
Some models, such as decision trees and ensembles of trees, can take a different form however when applied to n-class classification.
So the answer is highly dependent on the implementation of the algorithm you're using. But the baseline answer is that any multiclass classification algorithms can be used for 2-class classification, and most 2-class classification algorithms can be generalized for multi-class classification.
-
$\begingroup$ Wouldn't accuracy take a hit? (for multi-class classification). Also, is there a reasonable number of classes beyond which it might be a better idea to have a wide array of binary classifiers to get the same job done? $\endgroup$ Jul 21, 2015 at 0:37
-
$\begingroup$ You mean predicting each of the outcomes "is of class X" vs "is not of class X" ? this would be disastrous, as you would 1) find probabilities which do not normalize, 2) possibly have several "positive" classifications; 3) remove all the correlation structure between the various classes for prediction, thus actually increasing inaccuracy ; 4) have variance individual to each model rather than a single model, which usually means larger variance in final output $\endgroup$– YouloushJul 23, 2015 at 15:23