Consider a classification task wherein the training data has around 100,000 sentences with around 1000 labels. These 100,000 sentences can possibly be grouped hierarchically. The task at hand is given a random input string, find out the most similar of the 100,000 sentences and assign the corresponding label to the random input string.

The random input string may be only semantically related to one or more of the 100,000 labelled sentences and hence can be assigned one or more of the 1000 labels.

Also in the training data, the number of sentences of belonging to individual classes are highly imbalanced.

I have considered a lot of different ways but am not sure which would be the best way to go about it. The literature says that in general one should convert the problem to reduce the number of classes by following hierarchical classification. But has the drawback that the error rate cascades over the levels of hierarchy and results in poor overall performance

Other popular suggestion is convert the problem to one vs one similarity problem but this approach presents the issue of high class imbalance.

Any ideas about how to tackle this problem ?

  • 2
    $\begingroup$ What does "1 lac" mean? $\endgroup$
    – whuber
    Commented Aug 29, 2017 at 17:39
  • 2
    $\begingroup$ I'm with whuber regarding the meaning of "1 lac." Despite knowing a bit about text mining, NLP, etc., I've never seen this acronym used anywhere. Next, this suggestion doesn't come from classifying semantic "lacs," but is related and may be helpful. To your point about treating the classes hierarchically, some Bayesians developed an approach for modeling massively categorical information, e.g., zipcodes, which can have as many as 36,000 "classes" or levels. See a paper by Steenburgh and Ainslie, Massively Categorical Variables: Revealing the Information in Zip Codes. It's out there ungated. $\endgroup$
    – user78229
    Commented Aug 29, 2017 at 17:54
  • 1
    $\begingroup$ By 1 lac i mean 100,000. Edited that. Also i couldn't find any ungated links to the mentioned paper. $\endgroup$
    – m1cro1ce
    Commented Aug 29, 2017 at 18:43
  • 2
    $\begingroup$ @DJohnson The usual spelling is "lakh". (I couldn't discover this until I knew what quantity this word was intended to represent!) $\endgroup$
    – whuber
    Commented Aug 29, 2017 at 19:04
  • $\begingroup$ ad lakh: so OP actually means 1,00,000 sentences... :-) and thanks, DJohnoson for that paper hint, very interesting! $\endgroup$
    – fnl
    Commented Aug 30, 2017 at 7:08


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.