4
$\begingroup$

First let me explain what I mean by "transitive" Suppose that the price of product A and the price of product B has a correlation of .5 Suppose also that the price of product B and product C has a correlation of .5

One might thing that if correlation(A,B) = .5 and correlation(B,C) = .5 then correlation(A,C) > 0, but with an intuitive example we can see why this is not the case:

Suppose that A is a fruit salad made of papaya and banana, suppose also that B is a fruit salad made of banana and strawberry and, finally, suppose that C is strawberries and cream. Clearly, when the price of banana goes up, both the price of A and B will go up; when the price of strawberry goes up the price of B and C will go up; but since A and C have no ingredients in common, their price isn't correlated.

My question is which statistical concept captures this intuitive idea. Is it the concept of dimensions?

What do I mean by transitive? Let me define the binary operator X corr? Y as 1 if correlation(X,Y) != 0 and 0 otherwise, so:

  • A corr? B = 1,
  • B corr? C = 1, but
  • A corr? C could be = 0

If corr? was transitive, then A corr? C = 1

$\endgroup$
10
  • 1
    $\begingroup$ Pictures like given here can help a lot in understanding correlations or regressions among 3 variables. Correlation is equal to the cosine of angle between variable vectors. So, you may take 3 pen beam and experiment with the 3 angles. You will find that as r_AB and r_BC go towards 0 r_AC gets more room to variate. $\endgroup$
    – ttnphns
    Jan 16, 2017 at 7:32
  • $\begingroup$ Exactly what "intuitive idea" are you asking about? You mentioned "transitive" and gave an example, but you did not actually define "transitive" and your example does not comport with the standard meaning of transitivity. You have mentioned one specific statement about correlation and you have described one specific example of causal linkages, but if there's any underlying "idea" lurking here it doesn't seem to have been spelled out. Could you clarify? $\endgroup$
    – whuber
    Jan 16, 2017 at 21:33
  • 1
    $\begingroup$ it's not clear, in part because your example is not really about correlation: it's about causality. The two are completely different concepts. $\endgroup$
    – whuber
    Jan 16, 2017 at 22:02
  • 1
    $\begingroup$ That has been done many, many times by many people on this site. I most recently demonstrated it at stats.stackexchange.com/a/256131/919. You can find other demonstrations at stats.stackexchange.com/… . $\endgroup$
    – whuber
    Jan 16, 2017 at 22:11
  • 1
    $\begingroup$ @whuber, when you wrote that my question was about causality I was like “I don’t even want to know what he means”. Some years have passed, I’ve read Judea’s book The Book of Why and now I get it. Your comment was right, thanks! $\endgroup$ Oct 3, 2018 at 12:10

3 Answers 3

4
$\begingroup$

If the correlations of $A$ to $B$ and $B$ to $C$ are specified, then any value rho_AC such that the $3$ by $3$ matrix of correlations is positive semi-definite is a possible value of rho_AC. Given the correlations of $A$ to $B$ and $B$ to $C$, there will be a minimum and maximum such possible value of rho_AC.

Here is code in CVX under MATLAB which finds the minimum value of rho_AC. The maximum such value is achieved with the identical program, but using maximize rather than minimize. Although it is trivial to find the minimum and maximum possible values of rho_AC in this example, this approach still works quite well as the dimension of the problem increases.

% rho_AB and rho_BC are MATLAB variable values containing input correlations
cvx_begin
variable rho_AC
minimize(rho_AC)
% Next line constrains 3 by 3 correlation matrix to be positive semidefinite
[1 rho_AB rho_AC;rho_AB 1 rho_BC;rho_AC rho_BC 1] == semidefinite(3) 
cvx_end

Running this program for rho_AB -= 0.5 and rho_BC = 0.5, the minimum possible value for rho_AC is found to be -0.5 and the maximum possible value for rho_AC is found to be 1. Note that although this example is trivially solved without this fancy apparatus, this solution method readily and accurately (numerically stably) also works on higher dimensional problems and other more complicated variants.

Let the positive semi-definite cone be your guide to possible combinations of correlation. The bottom line is that the 3 dimensional positive semi-definite cone does not satisfy transitivity for its 3rd parameter with respect to the other 2 parameters. Note that when rho_AB and rho_BC are both non-negative but unequal to each other, the maximum value of rho_AC will be less than $1$.

Edit: With regard to your edit after my answer: Consider, for example, your example of corr(A,B) = corr(B,C) = 0.5, which I analyzed above. As shown above, per the mathematics of the positive semi-definite cone, $-0.5 \le$ corr(A,C) $\le 1$. In particular, corr(A,C) might be equal to $0$ or not.

$\endgroup$
1
$\begingroup$

I think it is less a different statistical concept than the amorphous nature of "correlation" itself. Dependence is usually defined as "not mathematically independent" even though we tend to conceptualize it the other way around.

In your example, you actually do not know if A & C are truly independent. For example, why did the price of papayas and bananas go up? If it was a drought in the area which also affected the strawberry crop or the grass fed to the cows who produced the milk for cream, there may be correlation.

There are many, many reasons why variables may not be independent. Dimensions as you say, and as ttnpnhs refers to in his comment is one way of showing it. Completely non-rigorously, it could be that the "planes" on which variables A & B and B & C interact are different enough so that the relationship between A & C, when forced into its plane, ends up with $P(A\cap C) = P(A)P(C)$ out of serendipity.

In a nutshell, I'd say that correlation is not transitive, because it it isn't a clean "function" or even a simple relationship. Rather, it is a shortcut definition comprising—or more accurately hiding—many possibilities. Therefore, outside of simple situations, it needs to be calculated manually for each pair of variables in question.

$\endgroup$
0
$\begingroup$

Transitivity is a property of binary relation. Correlation (e.g, Pearson correlation) is not a binary relation and therefore cannot be transitive.

One could define a binary relation using correlation by requiring correlation above a certain threshold. Your example presents that even with this definition, correlation is not transitive. The correlation strength can decay over a chain of relation and go bellow the threshold.

Note that if the variables are a multiplication of each other in a non zero scalar, than you have transitivity.

$\endgroup$
4
  • 2
    $\begingroup$ One correlation being greater than another is a binary relation. Thus this reply really doesn't seem to add any relevant information to the question. $\endgroup$
    – whuber
    Jan 16, 2017 at 17:17
  • $\begingroup$ In order to speak about the difference between correlation you should have two correlation and 3 variables. This is not a binary relation on the variables. As you wrote, it seems that the OP doesn't mean to the usual meaning of transitivity. $\endgroup$
    – DaL
    Jan 17, 2017 at 6:56
  • 2
    $\begingroup$ That comment belies some confusion. Let's check the definition given in the question. It describes $\mathcal{R}\subset V\times V$ on the set $V$ of all square-integrable random variables of nonzero variance (defined on some underlying probability space) via $$(X,Y)\in \mathcal{R}\text{ iff }\operatorname{Cor}(X,Y)\ne 0.$$ Because this is well-defined and is a subset of $V\times V$, it is a relation. (It obviously is symmetric and reflexive, btw.) By definition, it will be transitive exactly when $$(X,Y)\in\mathcal{R}\text{ and }(Y,Z)\in\mathcal{R}\text{ imply }(X,Z)\in\mathcal{R}.$$ $\endgroup$
    – whuber
    Jan 18, 2017 at 15:04
  • $\begingroup$ Correct. It wasn't defined so in the version of the question I saw but added later. $\endgroup$
    – DaL
    Jan 19, 2017 at 7:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.