I have a bunch of sentences that I want to do binary classification with SVM.

My sentences have varying lengths form 4 to 34. If I use word embeddings such as word2vec or skip gram to convert my words into word vectors, I would end up with matrices of very different sizes due to differences in sentence length.

What is the best way to get around that? I know that if I were to use a neural network classifier, I would just pad with zeroes and let the neural network figure out the features. But If I were to use a classical machine learning classification method, what is the best way to deal with sentences of varying lengths?


Have you checked doc2vec? Doc2vec returns a fixed representation for each document (sentence in your case), regardless of its length. Here is the original paper. There is also a Python implementation from the gensim package.

Another usual approach for text classification is to calculate the tf-idf matrix and use it as input to a classifier, in which case the columns of the matrix are the features. The tf-idf matrix represents the sentences in its rows and the unique terms (words) of all your sentences in its columns. Each element in the matrix is the tf-idf value of this sentence-term. You can find more information in the wiki page. scikit-learn has an implementation of tf-idf here.

  • $\begingroup$ Thank you for your answer! I have already tried Doc2Vec but the accuracy I get is not satisfactory. This is why I am trying other word embedding solutions. tf.idf would be a count based approach. I want to try embeddings $\endgroup$ – Lucia Dec 15 '18 at 15:51

For inputs of varying lengths I suggest to use an encoder/decoder architecture, such as an LSTM model. You can use pretrained encoders for example from the flair library https://github.com/zalandoresearch/flair .

Essentially you would be giving the model a sequence of words as input and it will generate a context vector (essentially the hidden layer of the LSTM). you can then use the context vector as input to a normal fully connected hidden layer and a softmax to do your classification. That's the theoretical setup. Whether that works or not remains to be explored :)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.