I have been digging in the concept of entropy for a while, now it comes to the implementation part I feel I am confused.
Imagine that we have a matrix 20 * 3 standing for 20 words 3 topics (by 20 words I mean the probability of those words in docs, and the topics as the column is just like clusters).
So the problem here is related to topic modeling on the text data.
Also, I qoute a part of the paper: "If the topic-word distribution p(t|w) is uniform, the word is not characteristic of any topic. If it is highly concentrated then it is. This can be captured using the inverse of the entropy H(w):"
Now I am unable to interpret, what conclusion I can derive if I say have 20 entropy? how can I result that the cluster is distinctive while it is a sum over all topics!?
I would like to see each word is characteristic of which topics, what can I do for that?
and if it can help this is the link of paper paper, and in
I appreciate if someone shed more light on the topic.