I have been trying to implement mutual information in a java program that tries to identify terms that occur often together in documents. Basically, the program does the following: There is a text collection of just over 4.000.000 summaries from Wikipedia. This is searched through based on a query, resulting in a collection of say 100 texts that are deemed as a match to the query by Lucene index. I then create bigrams (structures of two words) so that all words are paired with all the other words in the result collection. Each bigram is then calculated a mutual information value for. The idea is that the highest scoring bigrams should be (hopefully) directly relevant to the original query.
On to how I understand the Mutual Information equation, which for the record looks like this:
P(x,y) * log(P(x,y) / (P(x) * P(y)),
where P(x,y) is the probability of term x and y occurring in a document, P(x) is the probability of just term x occurs in a document, and likewise for P(y) for term y.
I am unsure if I understand correctly what this means in terms of calculating the probability values. It is calculated by finding the frequency of a term divided by the number of articles in the result set. But is it the term frequency or the document frequency for a given term?
Ie. is P(x) in a result set of 100 articles the number of times term x occurs in the collection divided by 100? Or is it the number of articles in which term x occurs at least once divided by 100? And if it is the former, how then is P(x,y) calculated?