# Pointwise mutual information for text using R

I have key-value pairs of text. The values can be multiple words (n-grams). For example,

A     abcd
A     efgh
B     abcd
C     wxyz
C     mnop


I want to calculate Pointwise Mutual Information for the pairs. Is there a function in R to do this? Otherwise, how can I go about it?

Thanks

## 1 Answer

There are many functions for estimating the mutual information or the entropy in R, for example the entropy package. Enter

install.packages("entropy")


at the R-prompt. You can then use the property that $pmi(x;y) = h(x) + h(y) - h(xy)$ to calculate the pointwise mutual information. You need to obtain frequency estimates for the two random variables first.

• This will give you the mutual information between the random variables, not the pointwise mutual information between the pairs. – rmalouf Jun 18 '13 at 17:31