0
$\begingroup$

So I'm working on a supervised text classification problem where I need to classify a 100 classes. The way I look at it, I have following approaches-

  1. 100 logistic regression classifications with each class against everything else; and then choose the class with highest logistic regression score

  2. A neural network with n (=3?) Layers with a 100 neurons (1 neuron per class) each. And then use softmax to get 100 probabilities and choose the one with the highest.

  3. Train a Random Forest Classifier? Not sure how good it will be with 100 classes.

Please add any other approaches you think would apply in this case, and let me know what you think will be the best approach here.

$\endgroup$
4
  • $\begingroup$ I'm willing to bet only a handful of features are important, so why not try something like a naive bayes classifier first go? These seem to work OK for text classification. I'd then maybe try a lasso or ridge logistic regression. $\endgroup$ Commented Sep 3, 2019 at 19:22
  • $\begingroup$ I really hope to get that granular (i.e. 100 classes) while classifying my text. Just want to see if I can get the desired results. Of course, there will be some overlap between my classes - which I can always merge later, but for now, I want to see if I can train a classifier to pick it up. $\endgroup$
    – kev
    Commented Sep 3, 2019 at 19:45
  • $\begingroup$ There is some overlap between your classes??? $\endgroup$
    – Dave
    Commented Sep 3, 2019 at 19:47
  • $\begingroup$ @Dave For example, one of the classes could be called "animal", and then another class would be "tiger" - basically a taxonomy. $\endgroup$
    – kev
    Commented Sep 4, 2019 at 3:28

1 Answer 1

3
$\begingroup$

About the question:
When you say "classifying 100 classes" I think you have 100 output categories. The VGG16 output had 1000 output categories, so it isn't unreasonable.

The question:
Given a problem whose output is the selection of 1 of 100 candidate classes, tell me about the machine learning tools that could be used to engage it, and their relative strengths and weaknesses.

Answering:
Your answers are much closer than you imagine. If you did it right you could make them nearly exactly the same except for the bootstrapping in the random forest.

Independent logistic regressions:

  • easy to make, understand, troubleshoot, and communicate by themselves.
  • Altogether might be a bit overwhelming.
  • requires enough but not google-big data to do a good job
  • not as good at understanding variable interactions across classes

Neural network with logistic then max indicator:

  • easier to code, straightforward to train
  • single monolith model
  • sometimes wants more data
  • sometimes long to train O(n^6)

Random Forest

  • single monolith model
  • less noisy output, targets mean, robust
  • does well with interactions
  • does well with moderate data

So I have to ask, what is the nature of your data (type, input element description, input ensemble description)? Is it words, numbers, images? Do you have 10, 40, or 100 samples per output category?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.