0
$\begingroup$

I have built a text classifier using OneClassSVM.

I have the training set which corresponds to only one label i.e("Yes") and I don't have the other("NO") label data. My task is to build a classifier which classifies the new unseen sentence(test data) as 1 if it is very similar to the training data. Else, it classifies as -1 i.e,(anomaly).

I have used Word2Vec to build the word embeddings for my training data. Then, I am using word-vector averaging with OneClassSVM to build a anomaly detector classifier.

This classifier is currently giving accuracy of about 50%-55%. I have to enhance this further to build a robust classifier.

Any suggestions to this problem would be helpful...

$\endgroup$
  • $\begingroup$ You would probably have to play around with SVM kernels, which is not trivial. Have you tried other methods, for ex. clustering? $\endgroup$ – user2974951 Sep 11 '18 at 8:54
  • $\begingroup$ Clustering didnot help. I tried all other kernels in SVM, but polynomial kernel worked well compared to other ones. $\endgroup$ – Naveen Y Sep 11 '18 at 9:32
  • $\begingroup$ Naive Bayes is also widely used for text classification, have you tried it? $\endgroup$ – user2974951 Sep 11 '18 at 9:35
  • 1
    $\begingroup$ NB works well when there is a labeled training data. But in my case I have the data of one label only. $\endgroup$ – Naveen Y Sep 11 '18 at 9:37
0
$\begingroup$

The question is than about your data - how representational your cases from training set are for the whole "yes" subset - ?

And what type of errors your classifier does?

You may also try to use word2vec to produce embeddings of the whole texts.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ My training data is about 7000 sentences in length. In this training set, I have some outliers in this set as well. For example, "We provide authorization to our users." is a sentence in my training set. I am crawling the web to get my test set. It's not performing well on my test set even after tuning the hyper-parameters. $\endgroup$ – Naveen Y Sep 24 '18 at 8:57
  • $\begingroup$ Sorry for not answering earlier, hectic time... Any luck with improving accuracy? If I understood you correctly you have 7000 sentences of class '1' and you pick a sentences from the web for tests. And what type of errors your classifier does more frequently - false positive or negative? 7000 sentence would have to be a very compact scope of meaning to be representative vs random sentence from the web... You should consider using pretrained embeddings. And ponder how representative is your training set for the scope of language (sentences) you want to classify as '1'. $\endgroup$ – MkL Oct 13 '18 at 10:47
  • $\begingroup$ I couldn't construct a confusion matrix in order to tell you whether model is making more false positives or negatives since i don't have labels in my data neither in my training set not in my test set. And I have also used pretrained embeddings to build a model using GoogleNews-vectors-negative300 , but that didn't work out well. $\endgroup$ – Naveen Y Oct 18 '18 at 7:11
  • $\begingroup$ Hm, how have you measured 50-50% accuracy? $\endgroup$ – MkL Oct 18 '18 at 20:33
  • $\begingroup$ Just by manual check on the predictions by the model. $\endgroup$ – Naveen Y Oct 19 '18 at 8:19
0
$\begingroup$

This paper Outlier Detection for Text Data discussed similar problem. I believe for a robust classifier you need to understand latent topics in the corpus, either with LSI approach as discussed in this paper or via a clustering approach in latent space. I think using de-noising autoencoder for learning features from sentence embedding is the most straight forward approach to obtain robust classifier.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.