I am working on a multi class text classification dataset which has 2719 data (only training set is available test set is not available) and 256 classes. Sentence length of each data is very short, average sentence length is 5 and many data has only single word. Issue I am facing is out of 256 classes 81 classes have only single data. I have tried Naive bayes with TF-IDF but the accuracy is very poor. Could some one suggest good approach to proceed ?


enter image description here

Thanks in advance

  • $\begingroup$ I think you have to utilize domain knowledge to this problem. You have to assume something about the classes themselves, and build rule that fit this knowledge (perhaps through a Bayesian framework). Can you give examples of classes? Is there any hierarchy or dependence you can assume on the classes? $\endgroup$ – tmrlvi Jul 9 '18 at 8:30
  • $\begingroup$ @tmrlvi data are not constrained to particular domain. Added some example data $\endgroup$ – Sathvi Siva Jul 9 '18 at 8:50
  • 1
    $\begingroup$ It seems that the classes are hierarchical. For example, make me a sandwich and are you male or female both fall under TM_PQ. (Maybe TM is a broader class). Let's follow with an example - if I ask are you male or what sex are you should it fall under TM_PQ_Are_You_Male_Or_Female? Or should it fall under a new class? For now it seems that there is a basic class, and you attach the sentence itself to make it a specific class. $\endgroup$ – tmrlvi Jul 9 '18 at 8:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.