I need some general advice and possible ideas.

Problem statement goes like this -- We are given a tweet and we have to specify associated labels for it like generalized hate, support, oppose, refutation, allegation, sarcasm.

The training data is ~6k tweets. However, there is very high class imbalance. Almost 90% of classes have 0s and rest are one.

The approach I have tried:

  • Classical BoW with a generic tweet preprocessing trained using SVM, XGB, Naive Bayes etc. Although accuracy is good (due to class imbalance) but AUC is very poor ~0.52.
  • Sophisticated techniques like LSTM, GRU with Glove embedding, BERT is performing even poorer AUC <0.49. In fact for best the classifier is predicting the same label for all test data. (I tried with Minority Oversampling too, it couldn't improve the performance either).

What I figured out that the BERT vocab is not recognizing most of the words and mapping it to zero.

What other approaches should I try? Any leads are appreciated.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.