Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.

For a matrix $M$ in which entries $m_{a,b}$ denote the number of co-occurrences between elements $a,b$ from two distinct sets $A$ and $B$, how do I identify pairs with a significantly high co-occurrence? The absolute number of occurrences differs a lot between different $a$ as well as between different $b$.

If I look for entries with high $p(a|b) = m_{a,b}/|m_b|$ with $|m_{.,b}|$ being the sum of all entries $m_{.,b}$, frequently occurring $a$s are favoured, and vice versa when only considering $p(b|a)$.

So far, my approach has been to compute $p(a|b) p(b|a)$, but I lose any intuition about what the resulting number really means, and I'm wondering whether there might be a better approach.

share|improve this question
Your terminology is somewhat foreign to me. Does this have something to do with probability and/or statistics? Does p(a|b) represent a conditional probability? Can you explain your terms for people like me who are not familiar with them? – Michael Chernick Sep 7 '12 at 22:48
Yes I guess I'm using the terminology a bit wrong given that my interest is a bit underspecified. But yes, p(a|b) would be the conditional probability of seeing a if we are seeing an occurrence of b. Think people of one country a visiting another country b. I'm looking for relevant (probably a better term than "significant") pairs of travel origins and destinations. If some countries have a high number of occurrences as origin or destination, the individual conditional probabilities aren't as meaningful as I'd like. – user979 Sep 8 '12 at 0:33

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.