16
$\begingroup$

This assertion was raised in the top response to this question. I think the 'why' question is sufficiently different that it warrants a new thread. Googling "exhaustive measure of association" did not produce any hits, and I'm not sure what that phrase means.

$\endgroup$
15
$\begingroup$

It might be best to understand "measure of association" in a multivariate distribution to consist of all properties that remain the same when the values are arbitrarily rescaled and recentered. Doing so can change the means and variances to any theoretically allowable values (variances must be positive; means can be anything).

The correlation coefficients ("Pearson's $\rho$") then completely determine a multivariate Normal distribution. One way to see this is to look at any formulaic definition, such as formulas for the density function or characteristic function. They involve only means, variances, and covariances--but covariances and correlations can be deduced from one another when you know the variances.

The multivariate Normal family is not the only family of distributions that enjoys this property. For example, any Multivariate t distribution (for degrees of freedom exceeding $2$) has a well-defined correlation matrix and is completely determined by its first two moments, also.

$\endgroup$
  • $\begingroup$ Am I right that according to the definition you are applying here, covariance would not be a measure of association? Since it would tend to expand as the variances expand. $\endgroup$ – user1205901 Jul 22 '15 at 6:49
  • 2
    $\begingroup$ That is correct. Although covariance obviously is related to a measure of association, it is not itself one because it is affected by other factors as well. $\endgroup$ – whuber Jul 22 '15 at 15:06
19
+50
$\begingroup$

Variates can be associated in ways that the Pearson correlation is completely blind to.

In the multivariate normal, the Pearson correlation is "exhaustive" in the sense that the only association possible is indexed by $\rho$. But for other distributions (even those with normal margins), there can be association without correlation. Here's a couple of plots of 3 normal random variates (x,y and x,z); they're highly associated (if you tell me the value of the $x$-variate, I'll tell you the other two, and if you tell me the $y$ I can tell you the $z$), but they are all uncorrelated.

enter image description here

Here's another example of associated but uncorrelated variates:

enter image description here

(The underlying point is being made about distributions, even though I'm illustrating it with data here.)

Even when the variates are correlated, the Pearson correlation in general doesn't tell you how -- you can get very different forms of association that have the same Pearson correlation, (but when the variates are multivariate normal, as soon as I tell you the correlation you can say exactly how standardized variates are related).

So the Pearson correlation doesn't "exhaust" the ways in which variates are associated -- they can be associated but uncorrelated, or they can be correlated but associated in quite distinct ways. [The variety of ways in which association not entirely captured by correlation can happen is quite large -- but if any of them happen, you can't have a multivariate normal. Note, however, that nothing in my discussion implies that this (that knowing $\rho$ defines the possible association) characterizes the multivariate normal, even though the title quote seems to suggest it.]

(A common way to address multivariate association is via copulas. There are numerous questions on site that relate to copulas; you may find some of them helpful)

$\endgroup$
  • $\begingroup$ Is there real world data with such distributions? $\endgroup$ – user14650 Aug 12 '15 at 8:54
  • $\begingroup$ @what Are there real world data even drawn from normal distributions? I doubt it, so (since my marginals were all normal in the diagrams) that would make the answer "no" immediately. The point of the examples was to show clearly why association between random variables is not as simple as sometimes assumed (how often do people calculate a Pearson correlation to measure association? Quite often), and also to point out that having normal margins and being multivariate normal are different. Very real examples where Pearson correlation doesn't capture what's going on certainly occur. $\endgroup$ – Glen_b Aug 12 '15 at 10:24
  • $\begingroup$ Let's not talk about distributions for a moment. When we calculate correlations from a dot cloud, we assume an underlying "geometrically shaped" (linear, hyperbolic, logarithmic, sine, etc.) ideal correlation from which the dots in the cloud deviate due to some "error". Now all the ideal shapes that I have seen abstracted from real data where continuous (without breaks) and always increasing along at least one axis (that is, not, for example, circular). My knowledge of data is limited, so I was wondering if there was in fact real world data whose correlation is non-continuous or circular. $\endgroup$ – user14650 Aug 12 '15 at 10:39
  • $\begingroup$ For example, there might be data that if I plot it will look like two dot clouds. If I blindly calculate correlations on this data, I might find one, while (or so I have been told) the plot clearly indicates that I am missing some unknown confounding variable which, if I accounted for it, would resolve the spurious relationship in my data. If my professor looked at your "x" or "y" shaped examples, he would tell me that I have two distinct subsets of data mixed up. $\endgroup$ – user14650 Aug 12 '15 at 10:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.