1
$\begingroup$

I'd like to compare two distributions of measurements for the same entities, to determine the degree to which the two distributions measure the same underlying phenomenon. For example, I'd like to compare countries in the Human Development Index to the same countries in the OECD Better Life Index to determine the degree to which the two indices measure the same thing.

I know that if I were comparing samples to see whether they came from the same population, I'd use, for example, the Kolmogorov–Smirnov test. In my case, though, I have additional information: the entity (country) in each distribution (index). Which test should I use that accounts for this additional information?

$\endgroup$
2
$\begingroup$

In psychometrics, the degree of agreement between two tests that are supposed to measure the same thing is called convergent validity. Agreement can be measured with Pearson correlation, if you think there's a linear relationship between the scores on the two measures, or with Spearman or Kendall correlation, if you just think there's a monotonic relationship. In this case the assumption of a linear relationship seems very unrealistic, because the two indices use different units that have no defined conversion factor, so I would recommend Spearman or Kendall.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Thank you! I'd not thought of Pearson/Spearman/Kendal. Appreciate you noting the concept, too, (convergent validity) as this gives me something to explore further. $\endgroup$ – Iain Dillingham Jul 14 '17 at 19:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.