Feature construction for text mining

Question

In the text mining, besides N-gram model, what are the state-of-art models for building feature space while capturing the dependence among the different words, or capturing the semantic meaning in the article?

Answer 1

Good question. In text mining extracting keywords (n-grams) alone cannot produce meaningful data nor discover “unknown” themes and trends.

Triples extraction from sentence is one of the key technique that has been discussed and used for feature construction. The aim here is to extract dependency relation from sentence i.e., extract sets of the form {subject, predicate[modifiers], object} out of syntactically parsed sentences.

Steps:

1) Get the syntactic relationship between each pair of words. 
2) Apply sentence segmentation to determine the sentence boundaries.
3) The Parser is then applied to generate output in the form of dependency relations, which represent the syntactic relations within each sentence. 

How this is different from n-gram?

Dependency relation allows the similarity comparison to be based on the syntactic relations between words, instead of having to match words in their exact order in n-gram based comparisons.

Examples & Further reading:

• Sentences - "Peter sent more than 20 email. However he didn't get a reply.
• POS Tags - "Peter/NNP" "sent/VBD" "more/JJR" "than/IN" "20/CD" "email/NN" "However/RB" "he/PRP" "did/VBD" "n't/RB" "get/VB" "a/DT" "reply/NN"
• Triples – "Peter: sent [more]: email" "he: get[n't][However]: reply"

The tree parser algorithm is explained in page 1, section 2.1 in the white paper http://ailab.ijs.si/delia_rusu/Papers/is_2007.pdf .

Available Tools

Stanford Parser and OpenNLP will help you to create dependency tree, however you'll have to code the logic to extract triples or sets of form.

Note

It is generally agreed that phrasal-level analysis is most suited for coarse but scalable text mining applications. Word-level analysis is noisy and lacks precision. Sentence-level is too structured and lacks practical applications. Semantic analysis requires a significant knowledge base or a domain lexicon creation effort.

Hope this helps.