Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I am working on understanding various document ranking algorithms like (TF-IDF, LSI, language models, etc) by actually implementing them. I want to understand LDA and using various resources to understand the algorithm. What I don't understand is how we come up with the latent (hidden) variables/topics. Can someone please explain it to me using examples like:

Doc1: "shipment of gold damaged in a fire",
Doc2: "delivery of silver arrived in a silver truck", 
Doc3: "shipment of gold arrived in a truck"}; 
Query: "gold silver truck"

I will really appreciate any help in this regard. Thanks!

share|improve this question
add comment

1 Answer

At a very high-level view, latent topics are formed from words that often appear together in the same documents.

Your examples don't have a clear set of topics, so let's use the following documents instead:

Doc1: After I eat my breakfast of apples, oranges, bananas, and grapes, I'm going to go snowboarding in the Alps if it's not too cold outside.
Doc2: Apples, oranges, bananas, and grapes make good smoothies.
Doc3: Apples, oranges, bananas, and grapes are tasty fruits.
Doc4: Snowboarding in the Alps is a lot of fun, but cold.
Doc5: My boyfriend lives in the Alps, where he teaches snowboarding.

Suppose we say there are two latent topics that we want to discover. The topics that we discover are likely to be:

  • Topic 1 (the "fruit" topic): represented most strongly by apples, oranges, bananas, grapes.
  • Topic 2 (the "Alps" topic): represented most strongly by Alps, snowboarding, cold.

Doc 1 is then about an equal mix of topic 1 and topic 2, docs 2-3 are mostly topic 1, docs 4-5 are mostly topic 2.

Here's an interesting example of latent dirichlet allocation applied to the WikiLeaks CableGate: http://idea.ed.ac.uk/topics/cables/browser/cables.html (The set of topics are on the left.)

Also, I wasn't sure if you wanted a high-level view or a more technical algorithmic explanation, so if it's the latter you were looking for, just say so and I can add a more technical explanation.

share|improve this answer
    
thanks for the response. Can you give a more technical algorithmic explanation. For example, Like given your Docs. how should I map words to documents, words to topics, etc. thanks! –  Ali Sep 8 '11 at 1:35
1  
Hi Raza, welcome to the site. This should have been left as a comment directly to raegtin's answer. A good way to show thanks is an upvote! Also if you are the same person whom asked the question, you should attempt to merge the accounts. –  Andy W Sep 8 '11 at 2:35
    
@Ali I've merged your two accounts. Please, don't forget to register your account. This will enable you to get notified by new responses/comments, earn enough rep to vote on response (>15) etc. –  chl Sep 8 '11 at 6:51
1  
@Ali There's a more technical explanation here: blog.echen.me/2011/08/22/… If you still have questions, feel free to ask more! –  raegtin Sep 8 '11 at 19:21
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.