Consider a classification task wherein the training data has around 100,000 sentences with around 1000 labels. These 100,000 sentences can possibly be grouped hierarchically. The task at hand is given a random input string, find out the most similar of the 100,000 sentences and assign the corresponding label to the random input string.

The random input string may be only semantically related to one or more of the 100,000 labelled sentences and hence can be assigned one or more of the 1000 labels.

Also in the training data, the number of sentences of belonging to individual classes are highly imbalanced.

I have considered a lot of different ways but am not sure which would be the best way to go about it. The literature says that in general one should convert the problem to reduce the number of classes by following hierarchical classification. But has the drawback that the error rate cascades over the levels of hierarchy and results in poor overall performance

Other popular suggestion is convert the problem to one vs one similarity problem but this approach presents the issue of high class imbalance.

Any ideas about how to tackle this problem ?

  • 2
    $\begingroup$ What does "1 lac" mean? $\endgroup$ – whuber Aug 29 '17 at 17:39
  • 2
    $\begingroup$ I'm with whuber regarding the meaning of "1 lac." Despite knowing a bit about text mining, NLP, etc., I've never seen this acronym used anywhere. Next, this suggestion doesn't come from classifying semantic "lacs," but is related and may be helpful. To your point about treating the classes hierarchically, some Bayesians developed an approach for modeling massively categorical information, e.g., zipcodes, which can have as many as 36,000 "classes" or levels. See a paper by Steenburgh and Ainslie, Massively Categorical Variables: Revealing the Information in Zip Codes. It's out there ungated. $\endgroup$ – Mike Hunter Aug 29 '17 at 17:54
  • 1
    $\begingroup$ By 1 lac i mean 100,000. Edited that. Also i couldn't find any ungated links to the mentioned paper. $\endgroup$ – m1cro1ce Aug 29 '17 at 18:43
  • 2
    $\begingroup$ @DJohnson The usual spelling is "lakh". (I couldn't discover this until I knew what quantity this word was intended to represent!) $\endgroup$ – whuber Aug 29 '17 at 19:04
  • $\begingroup$ ad lakh: so OP actually means 1,00,000 sentences... :-) and thanks, DJohnoson for that paper hint, very interesting! $\endgroup$ – fnl Aug 30 '17 at 7:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.