Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Give some examples where a simple correlation coefficient has a sign opposite to that of the corresponding partial correlation coefficient and comment on it.

It is a question from an examination paper.Please help.

share|improve this question
    
The following question is quite similar, and might be a good reference. stats.stackexchange.com/questions/44066/… –  Cam.Davidson.Pilon Nov 23 '12 at 21:50
add comment

2 Answers 2

up vote 2 down vote accepted

The sign of partial correlation coefficient is the same as the sign of linear regression coefficient. (In fact, partial $r$ is just one of the ways to standardize regressional $b$.) So, if we have some variables, for example three, $X$, $Y$, $Z$, and you want to know the sign of $r_{XY.Z}$ - the partial correlation between $X$ and $Y$ - you will be enough to know the sign of $b_X$ in regression of $Y$ by $X$ and $Z$ (or $b_Y$ in regression of $X$ by $Y$ and $Z$).

If we assume that the three variables are centered (their means were brought to 0), the formula of a linear regression coefficient found in many textbooks could be written as follows:

$b_X = \frac{SCP_{XY}SCP_{ZZ} - SCP_{ZY}SCP_{XZ}} {SS_XSS_Z - SCP_{XZ}^2}$

where SCP stands for "sum-of-crossproducts" and SS for "sum-of-squares". The denominator here is always positive, so the sign of $b_X$ depends entirely on the numerator. We can expand what is an SCP, for example $SCP_{XY}$:

$SCP_{XY} = \sqrt{SS_X}\sqrt{SS_Y}r_{XY}$

If we substitute all SCP in the numerator accordingly and then simplify we'll get that the numerator is proportional to the quantity

$r_{XY}-r_{ZY}r_{XZ}$

and its sign is the sign of this quantity. So, whatever the sign of zero-order correlation $r_{XY}$, the sign of partial correlation $r_{XY.Z}$ is determined by the last expression. Below is an example: $r_{XY}=.314$, $r_{YZ}=.589$, $r_{XZ}=.606$, $r_{XY.Z}=-.067$, negative because $.314-.589*.606<0$.

       X        Y        Z 

   1.339   -1.097     .014 
    .619    1.022     .792 
   -.722    1.127     .699 
   -.695   -1.081   -2.016 
   1.421     .318    1.068 
   1.467     .002    1.284 
   -.619     .692    -.691 
   -.319    1.228    2.002 
    .478   -1.056   -1.281 
    .490     .704    1.151 
   -.316    1.204     .030 
   -.203     .021    1.176 
    .168    1.732    1.741 
    .763    1.090    1.834 
   2.734    -.227    1.044 
  -1.603    -.447   -2.056 
   -.846    -.024    -.335 
   -.009     .132     .932 
   -.304     .118    -.938 
   -.612   -1.878   -1.655 
  -1.370    -.607    -.499 
   -.921    -.893   -1.136 
   -.534     .312    -.282 
   -.136   -1.189   -1.203 
    .406     .752     .338 
   -.069     .559    -.227 
    .534    -.547     .167 
   -.450     .417    -.512 
   1.364    1.319    1.327 
  -1.019     .190    -.157 
   1.608     .588     .861 
  -1.909    -.871   -1.322 
    .488    -.266     .361 
  -1.492   -1.645   -1.216 
    .533     .006     .791 
   -.341     .890     .939 
   -.862     .873    -.342 
  -2.076   -1.051   -1.160 
    .059    1.314    -.456 
   -.666    -.652   -1.761 
   -.742     .885     .606 
   -.333    -.087   -1.040 
    .789     .684    1.322 
   -.121    1.006     .766 
    .528    -.190     .206 
    .944    1.752    2.055 
   -.368    -.548    -.619 
   -.655     .432    -.141 
   -.663   -1.176   -1.164 
   -.799   -1.607   -1.844 
    .563    -.052    -.011 
   -.959   -1.281     .267 
   1.256     .323     .569 
   -.099     .869    -.693 
    .813   -1.057   -1.393 
   1.443    1.519    1.180 
   1.513    1.662    1.160 
   1.488     .494    -.285 
   -.247     .808     .324 
   -.903     .086    -.912 
    .750   -1.304     .717 
  -1.665    -.847   -1.045 
  -1.945    -.480    -.439 
    .105     .804    1.303 
   -.524    1.251    1.201 
   -.277   -1.400    -.391 
   -.936   -1.406    -.215 
   2.029     .318    1.128 
  -1.214    1.002   -1.313 
   -.180     .205    -.845 
   -.364    1.176    -.428 
   1.087    1.167    1.743 
   -.736    -.779   -1.038 
   -.386    1.176     .167 
    .022     .120    1.399 
    .749   -1.324    1.507 
   -.262    -.438   -1.634 
  -1.199    -.206    -.439 
   -.339   -1.687   -1.082 
  -1.529   -1.969   -1.179 
  -1.028    -.806   -1.331 
  -1.080   -1.855   -1.958 
    .072    -.523     .044 
   -.096     .481    -.214 
    .220    -.221     .931 
   1.217    -.801     .412 
  -1.542     .398    -.735 
  -1.238    1.301    -.361 
    .320     .806     .951 
   -.039    -.198    -.526 
    .588    -.001     .860 
   -.682   -1.109    -.607 
    .767    -.381     .255 
   -.783     .338     .475 
    .120    1.227     .345 
   -.207    -.607     .130 
   1.450    1.145     .721 
   -.903     .127     .646 
   1.567    1.106     .477 
    .382    -.942     .404
share|improve this answer
    
Thanks for answer.It is very helpful. –  Argha Nov 24 '12 at 14:57
    
I wonder if you have a typo in your top equation $b_x=SCP_{XY}SCP_{ZZ}...$, shouldn't that be: $b_x=SCP_{XY}SCP_{\boldsymbol{Y}Z}...$? –  gung Feb 16 '13 at 19:15
1  
@gung, No, no typo. $SCP_{ZZ}$ is correct (and is actually $SS_Z$) –  ttnphns Feb 16 '13 at 19:54
add comment

ttnphns gave a very good answer, but to complete the examination question .... I take it you want to know intuitively why the partial and simple autocorrelations could have opposite signs.

Consider the following fictional scenario. In a given town, people get fatter as they get older. As a consequence, their doctors recommend that they exercise more. So older (and fatter) people exercise more than young, skinny ones. The correlation between weight and exercise would be positive (simple correlation). But if you adjust for age, you would find that those who exercise have lower weight than those that do not exercise (for a given age).

Ignoring age distorts the effect of weight and exercise.

Something similar happens with categorical data, where it is called Simpson's paradox.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.