I have a multiclass text classification problem where I have very few documents for each class. The classes are imbalanced but I want to be able to predict the class when I have at least 200 - 300 documents. What methods do you suggest? Also the documents are from a specialized domain so I dont think transfer learning will work. Would you recommend word embeddings when working with specialized domain text?

  • $\begingroup$ What have you tried? $\endgroup$ – Demetri Pananos Sep 10 at 22:52

I would run a sequential neural network in python with keras: https://keras.io/getting-started/sequential-model-guide/

The idea is to vectorize your textes first with the words. For example:

I have a dream about yesterday
I want to sleep

Would give you the Words list: 'I, have, a, dream, about, yesterday, want, to, sleep' Then this words will be your features:

  I have a dream about yesterday want to sleep
1 1 1    1   1    1     1         0   0   0
2 1 0    0   0    0     0         1   1   1 

with your target y:

  classA classB
1  1      0
2  0      1

then you can remove all the stop words with some techniques, to reduce the amount of vectors in your algorithm, and then run the neural network. In the end you will have a big vector for every know class, where every word gets a number of occurence in that class, with a lot of zeros.

This will be memory expensive, but quite succesful.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.