I have a pool of thousands of binary vectors, all of the same length l.

I would like to sort them, by their correlation/similarity to a test vector v that also has length of l.

Also the vectors (test vector and the binary vector) are sparse - containing a majority of over 90% zeros while the test vector could be less sparse but also above 70%

Condensed example, v is the test vector:

v = [-1, 5, 0, 0, 10, 0, -7]
v1 = [1, 0, 0, 0 ,0 ,0 ,0]
v2 = [0, 1, 0, 0 ,1 ,0 ,0]
v3 = [1, 1, 0, 0 ,0 ,0 ,1]

I would expect to get here v2 for first place and and v3 for second, the reason why I want v3 to be second is that -7 matching 0 is more significant to 5 matching 1.

The logic behind what I'm looking for is that the higher the positive value is the more similar it would be for a 1 and the more negative a value is the more similar it would be to a 0.

Is there any method/correlation that is suited for this kind of purpose?

  • $\begingroup$ What is the formula behind "-7 matching 0 is more significant than (?) 5 matching 1"? This is not a normal correlation. $\endgroup$
    – Gijs
    Jan 23 '17 at 10:41
  • $\begingroup$ I know it's not that why I am having a problem with the solution... Simply put I want the bigger the absolute value is in the test vector to have more significance in the correlation calculation where negative values match zeros and positive to 1 - the example with -7 and +5 is to show that I want that -7 matching 0 is "stronger" than +5 matching 1 $\endgroup$ Jan 23 '17 at 11:55

Absolute value of Pearson correlation solved this problem


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.