My problem: The input data is a corpus of short documents (a few sentences each). In each document some expressions need to be classified to categories. A document must contain some categories (each expression has a single label), and the rest are optional. The task: given such an expression and its surrounding words, classify its category.

As a solution I thought to convert my vocabulary words to vectors using word2vec, and then apply some multi-class classifier.

Is there any classifier which is a particularly good fit to word2vec's output? I thought using svm, is there a recommended kernel?

  • 1
    $\begingroup$ Do you have training data where expressions are labelled by true categories? If so, I think you can use typical named entity recognition to solve it. The possible methods are like HMM, CRF, or even CRF+word2vec. $\endgroup$
    – Munichong
    Aug 24, 2016 at 16:55
  • $\begingroup$ Thanks, could you expand on CRF+w2v? The categories are interdependent, e.g. some of them usually appear only once, such that if an expression was labeled as A, further expressions aren't likely to be As. So, structure prediction seems to be in place. $\endgroup$
    – dimid
    Aug 25, 2016 at 4:41
  • $\begingroup$ Named entity recognition (NER) is to identify from text person names, places etc. It is like classifying expressions into predefined categories. So I think you can refer to some papers about NER. CRF+word2vec is just my guess. I do not know if there is any existing project has done it (I do not guarantee it works): CRF requires a feature vector for each word. So perhaps you can use w2v as the vector of a word. $\endgroup$
    – Munichong
    Aug 25, 2016 at 14:49

2 Answers 2


It is always hard to assess a priori the performance of a pre-treatment on the data. Even something as simple as normalizing the data does not have an obvious influence on the performance on the later trained classifiers (see per example this post : Normalizing data worsens the performance of CNN?).

However the following links may help you implement your idea :

Text Classification With Word2Vec the author assesses the performance of various classifiers on text documents, with a word2vec embedding. It happens that the best performance is attained with the "classical" linear support vector classifier and a TF-IDF encoding (the approach is really helpful in terms of code, especially if you work with python and sk-learn)

Regarding SVMs, there are kernels designed for text. I once had nice results with Information diffusion kernels and TF-IDF encoding. Or you have kernels that works directly on strings : Text Classification using String Kernels, but their implementations are scarcer...


The best place to start is with a linear kernel, since this is a) the simplest and b) often works well with text data. You could then try nonlinear kernels such as the popular RBF kernel.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.