Example where a simple correlation coefficient has a sign opposite to that of the corresponding partial correlation coefficient

Give some examples where a simple correlation coefficient has a sign opposite to that of the corresponding partial correlation coefficient and comment on it.

-
The following question is quite similar, and might be a good reference. stats.stackexchange.com/questions/44066/… – Cam.Davidson.Pilon Nov 23 '12 at 21:50

The sign of partial correlation coefficient is the same as the sign of linear regression coefficient. (In fact, partial $r$ is just one of the ways to standardize regressional $b$.) So, if we have some variables, for example three, $X$, $Y$, $Z$, and you want to know the sign of $r_{XY.Z}$ - the partial correlation between $X$ and $Y$ - you will be enough to know the sign of $b_X$ in regression of $Y$ by $X$ and $Z$ (or $b_Y$ in regression of $X$ by $Y$ and $Z$).

If we assume that the three variables are centered (their means were brought to 0), the formula of a linear regression coefficient found in many textbooks could be written as follows:

$b_X = \frac{SCP_{XY}SCP_{ZZ} - SCP_{ZY}SCP_{XZ}} {SS_XSS_Z - SCP_{XZ}^2}$

where SCP stands for "sum-of-crossproducts" and SS for "sum-of-squares". The denominator here is always positive, so the sign of $b_X$ depends entirely on the numerator. We can expand what is an SCP, for example $SCP_{XY}$:

$SCP_{XY} = \sqrt{SS_X}\sqrt{SS_Y}r_{XY}$

If we substitute all SCP in the numerator accordingly and then simplify we'll get that the numerator is proportional to the quantity

$r_{XY}-r_{ZY}r_{XZ}$

and its sign is the sign of this quantity. So, whatever the sign of zero-order correlation $r_{XY}$, the sign of partial correlation $r_{XY.Z}$ is determined by the last expression. Below is an example: $r_{XY}=.314$, $r_{YZ}=.589$, $r_{XZ}=.606$, $r_{XY.Z}=-.067$, negative because $.314-.589*.606<0$.

       X        Y        Z

1.339   -1.097     .014
.619    1.022     .792
-.722    1.127     .699
-.695   -1.081   -2.016
1.421     .318    1.068
1.467     .002    1.284
-.619     .692    -.691
-.319    1.228    2.002
.478   -1.056   -1.281
.490     .704    1.151
-.316    1.204     .030
-.203     .021    1.176
.168    1.732    1.741
.763    1.090    1.834
2.734    -.227    1.044
-1.603    -.447   -2.056
-.846    -.024    -.335
-.009     .132     .932
-.304     .118    -.938
-.612   -1.878   -1.655
-1.370    -.607    -.499
-.921    -.893   -1.136
-.534     .312    -.282
-.136   -1.189   -1.203
.406     .752     .338
-.069     .559    -.227
.534    -.547     .167
-.450     .417    -.512
1.364    1.319    1.327
-1.019     .190    -.157
1.608     .588     .861
-1.909    -.871   -1.322
.488    -.266     .361
-1.492   -1.645   -1.216
.533     .006     .791
-.341     .890     .939
-.862     .873    -.342
-2.076   -1.051   -1.160
.059    1.314    -.456
-.666    -.652   -1.761
-.742     .885     .606
-.333    -.087   -1.040
.789     .684    1.322
-.121    1.006     .766
.528    -.190     .206
.944    1.752    2.055
-.368    -.548    -.619
-.655     .432    -.141
-.663   -1.176   -1.164
-.799   -1.607   -1.844
.563    -.052    -.011
-.959   -1.281     .267
1.256     .323     .569
-.099     .869    -.693
.813   -1.057   -1.393
1.443    1.519    1.180
1.513    1.662    1.160
1.488     .494    -.285
-.247     .808     .324
-.903     .086    -.912
.750   -1.304     .717
-1.665    -.847   -1.045
-1.945    -.480    -.439
.105     .804    1.303
-.524    1.251    1.201
-.277   -1.400    -.391
-.936   -1.406    -.215
2.029     .318    1.128
-1.214    1.002   -1.313
-.180     .205    -.845
-.364    1.176    -.428
1.087    1.167    1.743
-.736    -.779   -1.038
-.386    1.176     .167
.022     .120    1.399
.749   -1.324    1.507
-.262    -.438   -1.634
-1.199    -.206    -.439
-.339   -1.687   -1.082
-1.529   -1.969   -1.179
-1.028    -.806   -1.331
-1.080   -1.855   -1.958
.072    -.523     .044
-.096     .481    -.214
.220    -.221     .931
1.217    -.801     .412
-1.542     .398    -.735
-1.238    1.301    -.361
.320     .806     .951
-.039    -.198    -.526
.588    -.001     .860
-.682   -1.109    -.607
.767    -.381     .255
-.783     .338     .475
.120    1.227     .345
-.207    -.607     .130
1.450    1.145     .721
-.903     .127     .646
1.567    1.106     .477
.382    -.942     .404

-
Thanks for answer.It is very helpful. – Argha Nov 24 '12 at 14:57
I wonder if you have a typo in your top equation $b_x=SCP_{XY}SCP_{ZZ}...$, shouldn't that be: $b_x=SCP_{XY}SCP_{\boldsymbol{Y}Z}...$? – gung Feb 16 '13 at 19:15
@gung, No, no typo. $SCP_{ZZ}$ is correct (and is actually $SS_Z$) – ttnphns Feb 16 '13 at 19:54

ttnphns gave a very good answer, but to complete the examination question .... I take it you want to know intuitively why the partial and simple autocorrelations could have opposite signs.

Consider the following fictional scenario. In a given town, people get fatter as they get older. As a consequence, their doctors recommend that they exercise more. So older (and fatter) people exercise more than young, skinny ones. The correlation between weight and exercise would be positive (simple correlation). But if you adjust for age, you would find that those who exercise have lower weight than those that do not exercise (for a given age).

Ignoring age distorts the effect of weight and exercise.

Something similar happens with categorical data, where it is called Simpson's paradox.

-
+1 for nice and clear example. – amoeba Mar 31 '15 at 14:46