Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to find a good feature selection method given the specific task.

However, even before feature selection, selection of the category set actually will determine the final result a lot. Let's say, in many cases, the categories are actually semi-flexible or totally flexible.

I can imagine, to get a better set of categories, both the purpose of text classification and the statistical feature of the text play important roles.

I want to know, is there any tutorial/discussion/suggestion on how to select the categories? And what methods can be applied to analyse the statistical features of the text collection? (I can think of using clustering.)

share|improve this question
What do you mean by categories? – Nick Oct 19 '11 at 0:33
@Nick By categories, I mean the classes/categories to which the text will be classified/categorized. E.g. 'sports', 'politics', etc. when one categorizes news articles. – Flake Oct 19 '11 at 7:38
There is a large body of work on topic modeling, which sounds like what you want. The most popular model is called Latent Dirichlet Allocation (LDA), there are also several extensions, the most relevant to your task would probably be "Correlated Topics Models". There is free software implementations of both on David Blei's website. – Nick Oct 19 '11 at 17:44
@Nick I did play a bit with LDA some time before, but I never thought about the link between LDA-topic inference and my problem. Thanks for this! – Flake Oct 19 '11 at 18:01

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.