Skip to main content
added 581 characters in body
Source Link
Megan
  • 43
  • 5

There are two binary vectors with 0s or 1s as values and the correlation between them is calculated; this is done for 20,000 pairs of vectors. Theoretically, is there a difference between having only 3 of these 20,000 correlations having perfect correlation vs. 15,000 of these 20,000 having perfect correlation? Can the perfect correlations in either of these scenarios be trusted more?

Is there a numerical method to actually test whether or not these perfect correlations are meaningful or trustworthy? For instance, after a certain sample size, could having only a couple perfect correlations out of thousands or millions of correlations be caused solely by chance?

Edit for more context:

I'm interested in comparing two binary vectors for utility meters (for instance, 0 = the meter is on, 1 = the meter is off). The correlations between these vectors are being used to see if these meters are most likely on the same transformers or not. So if we have 20,000 correlations between different meters and only 3 of them are perfect or high (0.9-1.0) correlations, can that be trusted versus maybe 15,000 of them being perfect or high correlations? And is there a numerical method in order to test if these perfect correlations are legitimate?

There are two binary vectors with 0s or 1s as values and the correlation between them is calculated; this is done for 20,000 pairs of vectors. Theoretically, is there a difference between having only 3 of these 20,000 correlations having perfect correlation vs. 15,000 of these 20,000 having perfect correlation? Can the perfect correlations in either of these scenarios be trusted more?

Is there a numerical method to actually test whether or not these perfect correlations are meaningful or trustworthy? For instance, after a certain sample size, could having only a couple perfect correlations out of thousands or millions of correlations be caused solely by chance?

There are two binary vectors with 0s or 1s as values and the correlation between them is calculated; this is done for 20,000 pairs of vectors. Theoretically, is there a difference between having only 3 of these 20,000 correlations having perfect correlation vs. 15,000 of these 20,000 having perfect correlation? Can the perfect correlations in either of these scenarios be trusted more?

Is there a numerical method to actually test whether or not these perfect correlations are meaningful or trustworthy? For instance, after a certain sample size, could having only a couple perfect correlations out of thousands or millions of correlations be caused solely by chance?

Edit for more context:

I'm interested in comparing two binary vectors for utility meters (for instance, 0 = the meter is on, 1 = the meter is off). The correlations between these vectors are being used to see if these meters are most likely on the same transformers or not. So if we have 20,000 correlations between different meters and only 3 of them are perfect or high (0.9-1.0) correlations, can that be trusted versus maybe 15,000 of them being perfect or high correlations? And is there a numerical method in order to test if these perfect correlations are legitimate?

Source Link
Megan
  • 43
  • 5

Method to test whether a small number of perfect correlations can be trusted within a large sample

There are two binary vectors with 0s or 1s as values and the correlation between them is calculated; this is done for 20,000 pairs of vectors. Theoretically, is there a difference between having only 3 of these 20,000 correlations having perfect correlation vs. 15,000 of these 20,000 having perfect correlation? Can the perfect correlations in either of these scenarios be trusted more?

Is there a numerical method to actually test whether or not these perfect correlations are meaningful or trustworthy? For instance, after a certain sample size, could having only a couple perfect correlations out of thousands or millions of correlations be caused solely by chance?