6
$\begingroup$

Most of the text classifiers are based on the bag-of-words approach where you loose the context that a particular word appears. As a solution (or simple solution?) we can use n-grams as features. But are there any classifiers which "gist" the idea and model it in someway before training?

Improve this question
$\endgroup$

4 Answers 4

Reset to default
4
$\begingroup$

I suggest two alternatives, that have been extensively used in Text Classification:

  • Using Latent Semantic Indexing, which consists of applying Singular Value Decomposition to the DocumentXTerm matrix in order to identify relevant (concept) components, or in other words, aims to group words into classes that represent concepts or semantic fields.
  • Using a lexical database like WordNet or BabelNet concepts in order to index the documents, allowing semantic-level comparison of documents. This approach is not statistical, and it faces a problem with Word Sense Disambiguation.

Both methods can be applied before training. None of the them aim at catching word order.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Could HMMs model this sort of a thing? $\endgroup$
    – samsamara
    May 19, 2014 at 11:17
  • 1
    $\begingroup$ Both HMMs and Latent Dirichlet Allocation (I add for having more extensive references) are, to my view, [generative] language models, extensively used in NLP, and in Text Classification as well. As language models, they aim to predict the next output symbol (word) in terms of the previous ones, so in fact they try to capture language order. So, in short, I believe they should do the job. LDAs are more trending nowadays. Alternatively, you may try deep learning, as multilayer neural networks can capture hidden relations among closely located words. And they are trending in NLP as well. $\endgroup$
    – Jose Maria Gomez Hidalgo
    May 20, 2014 at 8:09
2
$\begingroup$

The continuous word representation using Neural Networks is widely used to represent words. Surprisingly, it has the ability to model the semantic context of words, i.e. detect similar words and put them near together in feature space.

You can use the word2vect tool to process a large text corpus and create word vector. It is worth noting that for specific domain you need utilize a domain specific corpus for constructing word vectors.

Improve this answer
$\endgroup$
1
$\begingroup$

You should take a look at log-linear models; it's definitely a valid choice in your situation.

Improve this answer
$\endgroup$
1
$\begingroup$

API models exist which can achieve this.

https://askmacgyver.com/explore/program/universal-topic-classifier/5S2Q5x8K

It takes an array of categories or "bag of words" and a text string to analyze. It then returns a sorted percentage of relevance for this provided keywords.

Input Data

  {
    "text": "this bank provides an excelent service to its clients when opening a new account and with other operations",
    "classes": [
        "bank account",
        "online banking",
        "technical support",
        "mortgage",
        "retirement savings",
        "mutual funds",
        "student loan",
        "credit card",
        "financial news"
    ],
    "minCutOff": "0.001"
}

API Response

{
    "bank account": 0.6448822158372491,
    "technical support": 0.40099627067600924,
    "financial news": 0.28635987039897565,
    "mortgage": 0.2676284175575462,
    "student loan": 0.257628495744561,
    "online banking": 0.32395217514082025,
    "credit card": 0.2144582134037077,
    "mutual funds": 0.09250890827081894,
    "retirement savings": 0.13690496892541437
}
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.