binary and multiclass classifiers

Question

I have a simple yes/no problem so I was naturally inclined towards using a binary classifier because I was reading the following book http://ciml.info/dl/v0_9/ciml-v0_9-all.pdf and I quote from it:

[ Binary Classification: trying to predict a simple yes/no response. For instance, predict whether Alice will enjoy a course or not. Or predict whether a user review of the newest Apple product is positive or negative about the product.

Multiclass Classification: trying to put an example into one of a number of classes. For instance, predict whether a news story is about entertainment, sports, politics, religion, etc. Or predict whether a CS course is Systems, Theory, AI or Other. ]

But then my professor told me to try using naive Bayes and KNN for my problem which (according to the image I've attached below) are multiclass classifiers.

So can it be beneficial to use multiclass classifiers for a simple binary problem? I'm probably going to try a lot of classifiers to get the best accuracy but before going straight into practical tests I wanted to get some advice because due to the abundance of classifiers I'll have to pick a few(and choose the one providing the best results) and that choice must be based on a certain reasoning.

Answer 1

If you have only two classes it doesn't matter what is the classification rule (one-vs-all or mutually exclusive classes) since for two classes they are the same.

Answer 2

Don't think too much about multi-class vs extension to binary. It's really not that hard.

Some statistical methods are binary classification, e.g. logistic regression. We can still extend it to multi-class by fitting several times and combine them like in your diagram.

We also have some statistical algorithms that can handle multi-class by definition.

Answer 3

Before i answer your question, i would like to stress that i'm not agree with the bullet point "$a$ vs. $\neg a$" in the slide. Imagine you want to classify pictures according to the class "apples". You have dozens of training pictures of apples and nothing else that describes what is "not an apple". So if you must judge for an unseen picture whether if it belongs to the class $a =\textrm{apple}$ or $\neg a = \textrm{not an apple}$ you are not facing a binary but a unary classificastion problem (also known as one-class-classificastion problem).

Binary classification deals with the problem where two classes can be truly modelled in terms of existing traing samples or more concrete existing features. But if we have a case where we must find out if an objecgt belongs to a class or not, then we are not longer talking about discrimination of multiple classes but recognition of a single class.

And now lets focus on your question: binary is in fact a multiclass-classification problem. Even more, a binary (or a multiclass) classification problem, can be seen as a sequence of $n$ unary classification problems, where $n$ denotes the number of classes. There are plenty of papers on this topic. I suggest to read a little bit on it. I am sure that your yes/no prediction problem falls into this category.

One-class classification. Concept-learning in the absence of counter-examples

A Survey of Recent Trends in One Class Classification

One-Class versus Binary Classification: Which and When?