KDD 15 paper: scoring bigrams I am by no mean into Statistics nor NLP. And I am reading the following paper, trying to learn something new:
http://astro.temple.edu/~tuc17157/pdfs/grbovic2015kddA.pdf
The authors used a score threshold to pick up bigrams from the processed text. The formula of the score is (section 4.2):
$$score(w_i,w_j) = \frac{count(w_i \& w_j)}{count(w_i) count(w_j)}$$
My brain told me I have seen this expression before. After awhile, I recalled that:
$$\text{Two events are independent iff.  } P(A\&B) = P(A)P(B)$$
Then the score can be reduced to:
$$score(w_i, w_j) = \frac{count(w_i \& w_j)}{count(w_i) count(w_j)} \\= \frac{P(A\&B)}{P(A)P(B)} \div \text{number of total occurrences of all words}$$
Therefore the score is just a more convenient way of measuring the ratio of probabilities, since they only differ by a constant.

Does this mean if score is higher, the two words are more statistically dependent? How should I interpret this score statistically?

 A: Yes, higher score, more statistically dependent.  Note$$
\dfrac{\mathbb{P}(A,B)}{\mathbb{P}(A)\mathbb{P}(B)}
= \dfrac{\mathbb{P}(A|B)}{\mathbb{P}(A)}
= \dfrac{\mathbb{P}(B|A)}{\mathbb{P}(B)}
$$so it's a measure of how much more likely $A$ is, given $B$, than alone (or vice-versa).  If the ratio is greater than one, it means that words $A$ and $B$ are seen together more often than you'd expect at random, given their individual appearance rates.
When the counts are low, you need to take the ratio with a pinch of salt (because the probability estimates can change wildly with a single missed or additional observation).  However, that non-robustness probably doesn't matter a whole lot in the paper you cited, as the authors are only using the score to guide a data-extraction step.
Incidentally, I wondered if the authors might have smoothed their score (a traditional remedy to the robustness issue), but upon searching for "smooth" in the paper I found only the "smoothie" tag from their blog samples!
