2
$\begingroup$

I have a symmetric data matrix $A$, giving co-occurrence of events. That is, $A_{ij}$ is the frequency of occurrence of $i,j$ together. The diagonal elements of $A$ are unknown/indeterminate.

I am interested in detecting pairs of events $i,j$ that preferentially occur together.

The problem is that some rows/columns of this matrix are significantly larger than other rows/columns. The naive approach of selecting pairs with large $A_{ij}$ then ends up selecting these rows/columns. Is there a way to normalize rows/columns without losing the symmetry of the matrix, so that the fine detail of the matrix becomes visible?

$\endgroup$
3
  • 1
    $\begingroup$ How exactly are these "events" observed? This matters because it will influence the degree of dependence among the individual data entries. $\endgroup$
    – whuber
    Dec 15 '14 at 19:24
  • $\begingroup$ Isn't this an exact duplicate of your older question? $\endgroup$
    – whuber
    Dec 15 '14 at 19:25
  • $\begingroup$ @whuber See the older question for how the data is generated. It's the same matrix $A = f$ in both questions. Whether both questions are duplicates depends on the answer. Here I am just asking about a way to normalize the data so that the variability is better seen. The other question is about inferring correlations between events. Perhaps both questions can be answered using the same method... but we don't know that yet. $\endgroup$
    – becko
    Dec 15 '14 at 19:28
0
$\begingroup$

If event $i$ is very frequent then $A_{ij}$ will be relaitively large compared to $A_{k\ne i,j}$.

The solution is not to use naive approach. Instead convert your frequency matrix into the correlation matrix like suggested in this answer.

$\endgroup$
5
  • $\begingroup$ That's exactly the point. A very frequent event $i$ hides the variability of $A_{k\ne i,j}$. I do not understand matlab very well. Can you tell me the idea, just in formulas (I'll program it in my language) of what you're doing in that answer you linked? Also, note that in my data, the diagonal elements of $A$ are not present. $\endgroup$
    – becko
    Dec 15 '14 at 19:17
  • $\begingroup$ The answer you refer to assumes the $i$ and $j$ are numeric values, that the pairs are ordered, whereas these are unordered (that's what the symmetry of the matrix means), and that a Pearson correlation coefficient is needed, which does not seem to be relevant in this question. Thus it seems your solution is inapplicable. $\endgroup$
    – whuber
    Dec 15 '14 at 19:22
  • 1
    $\begingroup$ @whuber, you are right. Let me think of the way to apply the same idea to this problem. $\endgroup$
    – Aksakal
    Dec 15 '14 at 20:41
  • $\begingroup$ @Aksakal: Did you find a better solution? $\endgroup$ Aug 5 at 0:17
  • $\begingroup$ @kjetilbhalvorsen, didnt have a chance to think about it $\endgroup$
    – Aksakal
    Aug 5 at 0:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.