Finding words belonging to a topic Consider forum posts or any text where we'd be interested in finding out related words, given the data. What would be a solution for creating a topic cluster based on this data? E.g. We are interested in finding a pre-determined topic "Tools" (e.g. Wrench, Screwdriver, ...) with the least possible effort. Is there any software/method available?
I understand a simple term frequency solution will go a long way (find out which words occur relatively often in the topic compared to how often the word occurs outside of documents).
Note that this is really a different problem than saying "we are trying to find 5 clusters based on LDA" or "let's classify these documents into 5 as distinct as possible clusters".
Is there any way not to have to come up with the "tools" ourselves?
I was thinking perhaps something semi automatic like:


*

*Start with 1 object belonging to the topic "Tools" (e.g. "Wrench")

*Sort all unique words for all documents containing the word wrench based on how often they occur in the documents in comparison with how often they occur in the whole corpus (these words might potentially be Tools)

*Take the first X (magic number) unique words and 

*for these also consider newly "potential" documents, and reconsider which words might be belong to Tools


However, a severe limitation would be the fact that a story that might mention the word Wrench, will most likely not mention Screwdriver. So you definitely need a second step, where the words strongly correlating with Wrench might now be used instead to discover Screwdriver (if it would not stand out next to Wrench in documents)
Any ideas?
 A: One possible approach is PU learning, where PU means Positive-Undefined.  You search for texts which contains words like "wrench" and "screwdriver", manually check which of them are indeed about tools, and now you have some texts which are about tools (Positive set) and a lot of unlabeled texts (Undefined set).  The problem of building a classifier in such circumstances is called PU learning.
Another possible approach is called Interactive Machine Learning.  I am not sure if it is appliable here, but it is worth checking.
A: I have tried to extract similar common real world knowledge categories from Gutenberg Project with LDA, and -- as you insightfully observe -- the result was furthest from a neat discovery of real world categories.
I turned to lexical and semantic ontologies to pursue my own project and it proved very helpful. Since your main concern -- I guess -- is to have the knowledge available for practical applications, and their discovery is not your primary purpose, you can possible use the encoding that has already gone into these ontologies. (Again, if I have understood the purpose correctly...)
For example, regarding 'Tools', if you submit a vector of 'Tool' lemmas as seed words in Wordnet, and write a function to find their hypernyms and then each hypernym's hyponyms (co-hyponyms), you will end up with much more 'Tools' than you started with. 
Thus you can programmatically populate a small ontology for your needs in a relatively short time, definitely faster and more fun than encoding the categories manually. There are packages that integrate Wordnet with R and Python, and perhaps other languages, but these are the ones I know of, so perhaps you can easily adapt this approach to your workflow.
A: For your case, that is, an unsupervised problem, you could be using an approach known as query expansion, with your particular problem usually being referred to as Information Filtering. The idea is to let a system analyze the documents you want to work with, learn the usage of terms, and then, for a given query term, use that knowledge to "expand" the query with synonyms and related terms.
One possibility to do that is using Latent Semantic Indexing (LSI) on all (untagged) posts you have to identify/group synonymous and polysemous terms. Then assemble collections of terms for a topic of interest starting with a relevant term (such as the "tools" you describe) to assemble an "exanded query" for all documents with any of those terms and tag them with that topic using the similarity matrix LSI gave you.
A simpler way is to rank terms as found in the documents from in your initial query ("tools") against the posts or an independent set of documents (WikiPedia?) and use the highest ranked terms as your candidates, as described here.
