I have a labeled set of data which contains 10 classes and ~400 training examples for each class. I would like to develop a classifier using this data.
However, out of the 10 classes, I am only interested if something is
class 1 or
not class 1. Hence, I am unsure if I should simply create a binary classifier as opposed to creating a multi-class classifier.
I am also aware that each class should have roughly the same amount of training data as to not develop a skewed classifier.
Thus, if I want to develop a binary classifier I will need to use two classes:
class 1 and
not class 1, where each class has ~400 training examples. This will result in only using ~45 training examples from each of the nine classes that are not class 1 to develop the new
not class 1 class.
The problem is I am not sure if the act of using less data in my binary classifier will result in it being a worse classifier than if I created a multi-class classifier using all of my data.