7
$\begingroup$

My problem: The input data is a corpus of short documents (a few sentences each). In each document some expressions need to be classified to categories. A document must contain some categories (each expression has a single label), and the rest are optional. The task: given such an expression and its surrounding words, classify its category.

As a solution I thought to convert my vocabulary words to vectors using word2vec, and then apply some multi-class classifier.

Is there any classifier which is a particularly good fit to word2vec's output? I thought using svm, is there a recommended kernel?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Do you have training data where expressions are labelled by true categories? If so, I think you can use typical named entity recognition to solve it. The possible methods are like HMM, CRF, or even CRF+word2vec. $\endgroup$ – Munichong Aug 24 '16 at 16:55
  • $\begingroup$ Thanks, could you expand on CRF+w2v? The categories are interdependent, e.g. some of them usually appear only once, such that if an expression was labeled as A, further expressions aren't likely to be As. So, structure prediction seems to be in place. $\endgroup$ – dimid Aug 25 '16 at 4:41
  • $\begingroup$ Named entity recognition (NER) is to identify from text person names, places etc. It is like classifying expressions into predefined categories. So I think you can refer to some papers about NER. CRF+word2vec is just my guess. I do not know if there is any existing project has done it (I do not guarantee it works): CRF requires a feature vector for each word. So perhaps you can use w2v as the vector of a word. $\endgroup$ – Munichong Aug 25 '16 at 14:49
2
$\begingroup$

It is always hard to assess a priori the performance of a pre-treatment on the data. Even something as simple as normalizing the data does not have an obvious influence on the performance on the later trained classifiers (see per example this post : Normalizing data worsens the performance of CNN?).

However the following links may help you implement your idea :

Text Classification With Word2Vec the author assesses the performance of various classifiers on text documents, with a word2vec embedding. It happens that the best performance is attained with the "classical" linear support vector classifier and a TF-IDF encoding (the approach is really helpful in terms of code, especially if you work with python and sk-learn)

Regarding SVMs, there are kernels designed for text. I once had nice results with Information diffusion kernels and TF-IDF encoding. Or you have kernels that works directly on strings : Text Classification using String Kernels, but their implementations are scarcer...

Improve this answer
$\endgroup$
0
$\begingroup$

The best place to start is with a linear kernel, since this is a) the simplest and b) often works well with text data. You could then try nonlinear kernels such as the popular RBF kernel.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.