4
$\begingroup$

I am trying to locate a source/sources for this claim (from a reviewer):

I (and other measurement experts) believe that a correlation of .70 or higher indicates that two constructs are very much alike because at that point, two constructs share at least half of their variance. By definition, constructs that share this much variance are similar and potentially interchangeable.

The claim seems obviously wrong, simply because no statistic has a meaning independent of context. I've checked with two statisticians. Both agree with me and neither could provide a source for the idea. Does anyone have a source for this idea?

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ According to Quora, fruit flies share 60% of genes with humans, I guess fruit fly is interchangable with your reviewer. Moreover, in many disciplines with r=0.7 people would tell you to throw away this data. This is nonsense, I see no point in trailing such nonsenses, instead you should rather answer your reviewer that he is simply wrong and back it up with reasonable sources that support your claim. $\endgroup$
    – Tim
    Commented Jan 22, 2018 at 20:40
  • 2
    $\begingroup$ I've never heard of this, and the idea seems absurd, except for the weasel term "potentially". You might enjoy browsing through some of the counterexamples at tylervigen.com/spurious-correlations $\endgroup$
    – zbicyclist
    Commented Jan 22, 2018 at 21:30

2 Answers 2

Reset to default
1
$\begingroup$

I think this has to do with concurent validity. A related concept is multi-trait multi-analysis analysis. Note that in my reply to the linked question, I rather say that "the traits are defined in a consistent and objective manner." This means that if we were to administrate two different questionnaires that purport to measure the same construct to the very same sample, we expect individual scores to be highly correlated, pending minor issues like measurement error which can be mitigated using a dedicated correction. The choice of the threshold (0.7 or 0.8) is up to you, of course, like any such criteria. They are often arbitrary since it all depends on the sample size, the length and reliability of the questionnaires, etc.

Setting aside the shared variance that can be explained from simply decomposing the correlation between two latent variables, I think it remains tricky to affirm that "constructs that share this much variance are similar and potentially interchangeable." First, you can reach a very high correlation with constructs that are not directly related, i.e. traits that can be found in isolation or combined (in health measurement, for instance, we used to think that measuring quality of life is essentially measuring depression); second, there's a gross approximation between constructs and scale scores (whether your scale effectively measure what it claims to has to do with construct validity), IMO. I believe these two points are in line with @Tim's comments.

Improve this answer
$\endgroup$
0
$\begingroup$

As written, interpreted literally, the quote seems permissible. It isn't as strong as the claim in the title of your question, and "interchangeable" is hedged with "potentially". Of course, there's nothing magic about a .7 correlation or half the variance. The same argument applies to a lesser or greater degree as you change the degree of association. And the same correlation will appear more or less impressive depending on context, in line with your point that "no statistic has a meaning independent of context".

Improve this answer
$\endgroup$
3
  • $\begingroup$ Sorry but I don't agree, how is it permissible? Such correlation suggests that the variables are "somehow similar", but certainly not "interchangeable". $\endgroup$
    – Tim
    Commented Jan 22, 2018 at 20:46
  • $\begingroup$ @Tim Perhaps I'm being overly generous in my interpretation of the word "interchangeable". I definitely wouldn't expect the two variables to interchangeable for every purpose, but it's not hard to imagine finding that they would have roughly the same coefficient when one is replaced with the other in a regression model—perhaps the common variance is what's driving the coefficient. $\endgroup$
    – Kodiologist
    Commented Jan 22, 2018 at 20:50
  • $\begingroup$ @Tim I believe the context here is very important, the language here sounds like this is psychology. If so, there are extraordinary errors in many measurements (compared to other fields) such that r=0.7 is indeed a reasonably expected relationship between different measures of the same latent variable; this is the reason for the word 'interchangeable' which sounds far too strong but may not be. It's still a rather unhelpful review comment, but I suspect the reviewer simply wants the author to discuss the possibility that two measures in the paper estimate the same latent variable. $\endgroup$
    – Bryan Krause
    Commented Jan 23, 2018 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.