3
$\begingroup$

In the scenario that I have a binary classification problem, and use a binary classifier to train and test my model, assuming everything else is constant, would using a multi-class classifier with 2 labels (one vs. all) produce the same results? (I know I should use the binary classifier, but I'm simply curious).

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

"Binary classification" is simply multi-class classification with 2 labels. However, several classification algorithms are designed specifically for the 2-class problem, where the response is modeled as the outcome of a Bernouilli trial.

Of course, this can be generalized to the n-dimensional setting by modeling as the outcome of a multinomial sample. For instance logistic regression can be generalized to multinomial logistic regression.

Some models, such as decision trees and ensembles of trees, can take a different form however when applied to n-class classification.

So the answer is highly dependent on the implementation of the algorithm you're using. But the baseline answer is that any multiclass classification algorithms can be used for 2-class classification, and most 2-class classification algorithms can be generalized for multi-class classification.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Wouldn't accuracy take a hit? (for multi-class classification). Also, is there a reasonable number of classes beyond which it might be a better idea to have a wide array of binary classifiers to get the same job done? $\endgroup$
    – user799188
    Jul 21, 2015 at 0:37
  • $\begingroup$ You mean predicting each of the outcomes "is of class X" vs "is not of class X" ? this would be disastrous, as you would 1) find probabilities which do not normalize, 2) possibly have several "positive" classifications; 3) remove all the correlation structure between the various classes for prediction, thus actually increasing inaccuracy ; 4) have variance individual to each model rather than a single model, which usually means larger variance in final output $\endgroup$
    – Youloush
    Jul 23, 2015 at 15:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.