I have key-value pairs of text. The values can be multiple words (n-grams). For example,

A     abcd
A     efgh
B     abcd
C     wxyz
C     mnop

I want to calculate Pointwise Mutual Information for the pairs. Is there a function in R to do this? Otherwise, how can I go about it?



There are many functions for estimating the mutual information or the entropy in R, for example the entropy package. Enter


at the R-prompt. You can then use the property that $pmi(x;y) = h(x) + h(y) - h(xy) $ to calculate the pointwise mutual information. You need to obtain frequency estimates for the two random variables first.

  • 1
    $\begingroup$ This will give you the mutual information between the random variables, not the pointwise mutual information between the pairs. $\endgroup$ – rmalouf Jun 18 '13 at 17:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.