Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a simple (may be stupid) question. I want to calculate KullbackÔÇôLeibler divergence on two documents. It requires probability distribution of each document.

I do not know how to calculate probability for each document. Any simple answer with layman example would be much appreciated.

Let's say we have follow two documents:

1 - cross validated answers are good 
2 - simply validated answers are nice

(wording of the documents is just bla bla to give you an example)

How do we calculate probabilities for these documents?

Let's say we add one more document:

3 - simply cross is not good answer

If we add another document then how would it impact probability distribution?


share|improve this question
maybe related:… – steffen Jul 6 '11 at 11:33
No, it is not duplicate. They never explained with any layman example and I don't want to use any LDA or any other approach. I want to know how to calculate without any approach. – user3900 Jul 6 '11 at 13:31
I did not say it is a duplicate :). "I want to know how to calculate without any approach." What you do mean ? As far as I understand your question, a layman explanation of LDA would be fine. Or not ? – steffen Jul 6 '11 at 13:36
Thank you for your reply :). LDA is full of math. I just want to know if you documents (same like above). How can you calculate probabilities on a paper :-). – user3900 Jul 6 '11 at 15:10
What is the probability distribution as it relates to the document? This is something you have to define. What are you trying to do exactly? – Emre Jul 7 '11 at 17:05

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.