I have 10 different classes in my classification problem. Each class has about 200 instances, with more than 10.000 features. I performed the classification using Multinomial Bayes classification. However, it turned out that I'm only interested in the results of the 10th category.

My first thought was too merge categories 1 to 9 as negative instances, while making category 10 the positive instances, effectively changing it to a binary problem. However, my advisor tells me that multi-class classification might be better, since there are a lot of features that specifically identify a certain category, while that might not be the case if all instances of category 1 till 9 are thrown together.

I find it hard to fathom that the results will not improve if we change it to a binary classification. Multi-class classification always seems more difficult than a simple binary classification.

What are the general opinions on this? Are there cases were multi-class classification returned better results for a specific class than for using binary classification for that class? I just want to know if this is a possibility in general, so I can defend my choices in the paper. If there are any nice papers/resources about this problem out there, please point towards them!


1 Answer 1


This is actually true as it is possible from this simulated example using R

sigma <- matrix(c(1,0,0,1), ncol=2)
x1 <- rmvnorm(n=500, mean=c(0,0), sigma=sigma, method="chol")
x2<- rmvnorm(n=500, mean=c(3,0), sigma=sigma, method="chol")
x3 <- rmvnorm(n=500, mean=c(1.5,3), sigma=sigma, method="chol")
x4 <- rmvnorm(n=500, mean=c(-2.5,3), sigma=sigma, method="chol")
x5 <- rmvnorm(n=500, mean=c(-4,-2), sigma=sigma, method="chol")

Visualize the data


Let's fit the first model and see accuracy and a plot of the predicted

 fit1<-naiveBayes(factor(class) ~., data, laplace = 0)
 [1] 0.9228

Now change the data and repeat the same steps for the second model with binary classification


 fit2<-naiveBayes(factor(class) ~., data2, laplace = 0)

The underlying reason is that having a distribution for each class enhance the flexibility and can model regions with different shapes


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.