I have around 20,000 manually categorized texts into 500 classes. Around 150 of the classes have only one instance in the data. If I limit it to classes with at least 4 instances in the data, I get to 275 classes. I have trained a deep learning model based on the 275 classes (everything else classified as "other" in the training set). The model works relatively well, however, it is of interest to include as many classes as possible. Is there a criterion or at least a rule of thumb for the minimum number of instances from each class in a multi-class classification by deep learning? Are there any reasons to not include all the classes, including the ones with 1 instance, in the model training?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
There is no rule of thumb, it very much depends on how difficult your dataset is.
f you have only one instance for a class, the model will certainly have the capacity to memorize that particular instance. If you have four of them, it will very likely happen as well. Including those in the training data probably would not do much harm, but you cannot expect the classes to be ever predicted by the model.
I would suggest the following experiment: Take one class for which you have enough data, gradually remove the class instances from the training data, train a model with the modified training data and measure precision and recall for that class on validation data. You will see at which number the classifier starts to fail on that particular class.
