1
$\begingroup$

There are two binary vectors with 0s or 1s as values and the correlation between them is calculated; this is done for 20,000 pairs of vectors. Theoretically, is there a difference between having only 3 of these 20,000 correlations having perfect correlation vs. 15,000 of these 20,000 having perfect correlation? Can the perfect correlations in either of these scenarios be trusted more?

Is there a numerical method to actually test whether or not these perfect correlations are meaningful or trustworthy? For instance, after a certain sample size, could having only a couple perfect correlations out of thousands or millions of correlations be caused solely by chance?

Edit for more context:

I'm interested in comparing two binary vectors for utility meters (for instance, 0 = the meter is on, 1 = the meter is off). The correlations between these vectors are being used to see if these meters are most likely on the same transformers or not. So if we have 20,000 correlations between different meters and only 3 of them are perfect or high (0.9-1.0) correlations, can that be trusted versus maybe 15,000 of them being perfect or high correlations? And is there a numerical method in order to test if these perfect correlations are legitimate?

Improve this question
$\endgroup$
4
  • $\begingroup$ Your intuition is good, but I think the question as it stands is too abstract yet non-technical for me to give a good answer. Can you edit the question to provide more context? What set you to think about this, and what larger problem are you trying to solve? $\endgroup$
    – Kodiologist
    Commented May 18, 2016 at 20:15
  • $\begingroup$ Could you explain what you mean by a correlation to be "legitimate" or "trusted"? Do you have data quality concerns, perhaps? Your underlying problem is of interest because it is the same one shared, for example, by standardized testing companies. A multiple-choice standardized test can be considered a binary vector (1 when an answer is marked, 0 for all other possible answers). Strong "correlations" can be taken as evidence of copying. The literature on testing might give you some useful ideas about good ways to assess all these pairs and how to decide which ones are from common transformers. $\endgroup$
    – whuber ♦
    Commented May 18, 2016 at 22:03
  • $\begingroup$ Some meters are incorrectly labeled as being on the same transformer when they aren't or vice versa, so the correlations are being done in order to find out which of these meters are being incorrectly labeled. Thank you for the reference! I'll look into standardized testing. $\endgroup$
    – Megan
    Commented May 19, 2016 at 12:59
  • $\begingroup$ @whuber do you happen to have any sources for the literature on testing or possibly other examples I can look into? I've read all the academic papers on testing I could find or that I could access, but I haven't come across any that treat the standardized tests as binary vectors to detect cheating. $\endgroup$
    – Megan
    Commented May 19, 2016 at 15:41

1 Answer 1

Reset to default
1
$\begingroup$

It is a little beyond what statistics can do to tell you if an observation is "legitimate" or "can be trusted", because that's a very context-specific matter. Among the things it can do that are sort of like that is quantify how far an observation is from the rest of the data; that is, how much of an outlier it is. All other things being equal, a data point that's more of an outlier is more likely to represent bad data of some kind (anything from a number being written down incorrectly, to a rich person ending up in a sample that you wanted to include only poor people). If you want to look for outliers, the first thing to do is plot your data. (Actually, the first thing to do in pretty much any data-analysis problem is plot your data.)

However, I don't think quantifying outliers is necessarily a good way, on its own, to answer your question about whether different meters are on the same transformer. For that, you presumably want a sample of distinct pairs of meters that you know aren't on the same transformer, and a separate sample of distinct pairs of meters that are. Then you can use the data from these known pairs to classify the unknown pairs in your original data.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you for your response! I thought whether or not the perfect correlations were legitimate would be based on the knowledge of the data, but I was unsure if there was actually any way to test for it. $\endgroup$
    – Megan
    Commented May 19, 2016 at 13:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.