I am working on understanding various document ranking algorithms like (TF-IDF, LSI, language models, etc) by actually implementing them. I want to understand LDA and using various resources to understand the algorithm. What I don't understand is how we come up with the latent (hidden) variables/topics. Can someone please explain it to me using examples like:

Doc1: "shipment of gold damaged in a fire",
Doc2: "delivery of silver arrived in a silver truck", 
Doc3: "shipment of gold arrived in a truck"}; 
Query: "gold silver truck"

I will really appreciate any help in this regard. Thanks!


1 Answer 1


At a very high-level view, latent topics are formed from words that often appear together in the same documents.

Your examples don't have a clear set of topics, so let's use the following documents instead:

Doc1: After I eat my breakfast of apples, oranges, bananas, and grapes, I'm going to go snowboarding in the Alps if it's not too cold outside.
Doc2: Apples, oranges, bananas, and grapes make good smoothies.
Doc3: Apples, oranges, bananas, and grapes are tasty fruits.
Doc4: Snowboarding in the Alps is a lot of fun, but cold.
Doc5: My boyfriend lives in the Alps, where he teaches snowboarding.

Suppose we say there are two latent topics that we want to discover. The topics that we discover are likely to be:

  • Topic 1 (the "fruit" topic): represented most strongly by apples, oranges, bananas, grapes.
  • Topic 2 (the "Alps" topic): represented most strongly by Alps, snowboarding, cold.

Doc 1 is then about an equal mix of topic 1 and topic 2, docs 2-3 are mostly topic 1, docs 4-5 are mostly topic 2.

Here's an interesting example of latent dirichlet allocation applied to the WikiLeaks CableGate: http://idea.ed.ac.uk/topics/cables/browser/cables.html (The set of topics are on the left.)

Also, I wasn't sure if you wanted a high-level view or a more technical algorithmic explanation, so if it's the latter you were looking for, just say so and I can add a more technical explanation.

  • $\begingroup$ thanks for the response. Can you give a more technical algorithmic explanation. For example, Like given your Docs. how should I map words to documents, words to topics, etc. thanks! $\endgroup$
    – Ali
    Commented Sep 8, 2011 at 1:35
  • 1
    $\begingroup$ Hi Raza, welcome to the site. This should have been left as a comment directly to raegtin's answer. A good way to show thanks is an upvote! Also if you are the same person whom asked the question, you should attempt to merge the accounts. $\endgroup$
    – Andy W
    Commented Sep 8, 2011 at 2:35
  • $\begingroup$ @Ali I've merged your two accounts. Please, don't forget to register your account. This will enable you to get notified by new responses/comments, earn enough rep to vote on response (>15) etc. $\endgroup$
    – chl
    Commented Sep 8, 2011 at 6:51
  • 1
    $\begingroup$ @Ali There's a more technical explanation here: blog.echen.me/2011/08/22/… If you still have questions, feel free to ask more! $\endgroup$
    – raegtin
    Commented Sep 8, 2011 at 19:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.