finding significant pairs in co-occurrence matrix

Question

For a matrix $M$ in which entries $m_{a,b}$ denote the number of co-occurrences between elements $a,b$ from two distinct sets $A$ and $B$, how do I identify pairs with a significantly high co-occurrence? The absolute number of occurrences differs a lot between different $a$ as well as between different $b$.

If I look for entries with high $p(a|b) = m_{a,b}/|m_b|$ with $|m_{.,b}|$ being the sum of all entries $m_{.,b}$, frequently occurring $a$s are favoured, and vice versa when only considering $p(b|a)$.

So far, my approach has been to compute $p(a|b) p(b|a)$, but I lose any intuition about what the resulting number really means, and I'm wondering whether there might be a better approach.

Answer 1

It looks like you might be looking for a measure such as mutual information, log likelihood or dice. There are also a number of recent papers that look at this. See This stack overflow post using the checkerboard score.

Mutual information is best for unlikely pairs and looks for the difference between the product of the two marginal distribution and the joint distribution. https://en.wikipedia.org/wiki/Mutual_information

dice and log likelihood may work better for your case if you are looking for more common pairings. Dice is calculated by by taking the joint distribution over the sum of the two marginal distributions and then multiplying by two.

There is another website that covers this in detail with reference to word co occurrences and there is a large amount of discussion of this topic in this area of research. https://tm4ss.github.io/docs/Tutorial_5_Co-occurrence.html#3_statistical_significance