14
$\begingroup$

This is probably demonstrating a fundamental lack of understanding of how partial correlations work.

I have 3 variables, x,y,z. When I control for z, the correlation between x and y increases over the correlation between x and y when z was not controlled for.

Does this make sense? I tend to think that when one controls for the effect of a 3rd variable, the correlation should decreases.

Thank you for your help!

$\endgroup$
  • $\begingroup$ I can't top what probabilityislogic has done, but for a light treatment that gives illustrative examples and requires no math, see integrativestatistics.com/partial.htm $\endgroup$ – rolando2 Apr 27 '11 at 1:35
7
$\begingroup$

Looking at the wikipedia page we have the partial correlation between $X$ and $Y$ given $Z$ is given by:

$$\rho_{XY|Z}=\frac{\rho_{XY}-\rho_{XZ}\rho_{YZ}}{\sqrt{1-\rho_{XZ}^{2}}\sqrt{1-\rho_{YZ}^{2}}}>\rho_{XY}$$

So we simply require

$$\rho_{XY}>\frac{\rho_{XZ}\rho_{YZ}}{1-\sqrt{1-\rho_{XZ}^{2}}\sqrt{1-\rho_{YZ}^{2}}}$$

The right hand side has a global minimum when $\rho_{XZ}=-\rho_{YZ}$. This global minimum is $-1$. I think this should explain what's going on. If the correlation between $Z$ and $Y$ is the opposite sign to the correlation between $Z$ and $X$ (but same magnitude), then the partial correlation between $X$ and $Y$ given $Z$ will always be greater than or equal to the correlation between $X$ and $Y$. In some sense the "plus" and "minus" conditional correlation tend to cancel out in the unconditional correlation.

UPDATE

I did some mucking around with R, and here is some code to generate a few plots.

partial.plot <- function(r){  
  r.xz<- as.vector(rep(-99:99/100,199))  
  r.yz<- sort(r.xz)  
  r.xy.z <- (r-r.xz*r.yz)/sqrt(1-r.xz^2)/sqrt(1-r.yz^2)  
  tmp2 <- ifelse(abs(r.xy.z)<1,ifelse(abs(r.xy.z)<abs(r),2,1),0)  
  r.all <-cbind(r.xz,r.yz,r.xy.z,tmp2)  
  mycol <- tmp2  
  mycol[mycol==0] <- "red"  
  mycol[mycol==1] <- "blue"  
  mycol[mycol==2] <- "green"  
  plot(r.xz,r.yz,type="n")  
  text(r.all[,1],r.all[,2],labels=r.all[,4],col=mycol)  
}

so you submit partial.plot(0.5) to see when a marginal correlation of 0.5 corresponds to in partial correlation. The plot is color coded so that red area represents the "impossible" partial correlation, blue area where $|\rho|<|\rho_{XY|Z}|<1$ and the green area where $1>|\rho|>|\rho_{XY|Z}|$ Below is an example for $\rho_{XY}=r=0.5$

Partial correlation when marginal correlation is 0.5

$\endgroup$
  • $\begingroup$ The code does not work for me. Line 5 looks strange and Emacs tells me that some parentheses are missing. $\endgroup$ – Bernd Weiss Apr 26 '11 at 10:50
  • $\begingroup$ There's a missing "return" - must have accidentally deleted it. should be good now. $\endgroup$ – probabilityislogic Apr 26 '11 at 11:00
  • $\begingroup$ Thanks, works now! However, I am having some difficulties to understand the plot: is it really $r_{xz}$ on both axes? $\endgroup$ – Bernd Weiss Apr 26 '11 at 11:31
  • $\begingroup$ dog gone it! need to fix my code... again - plot() is wrong. Ah the glorious tedium of computer code $\endgroup$ – probabilityislogic Apr 26 '11 at 12:19
3
$\begingroup$

I think that variable z in the question is a suppresor variable.

I suggest having a look at:

Tzelgov, J., & Henik, A. (1991).Suppression situations in psychological research: Definitions, implications, and applications, Psychological Bulletin, 109 (3), 524-536. http://doi.apa.org/psycinfo/1991-20289-001

See also: http://dionysus.psych.wisc.edu/lit/articles/TzelgovJ1991a.pdf

HTH, dror

$\endgroup$
2
$\begingroup$

I think you need to know about moderator and mediator variables. The classic paper is Baron and Kenny [cited 21,659 times]

A moderator variable

"In general terms, a moderator is a qualitative (e.g., sex, race, class) or quantitative (e.g., level of reward) variable that affects the direction and/or strength of the relation between an independent or predictor variable and a dependent or criterion variable. Specifically within a correlational analysis framework, a moderator is a third variable that affects the zero-order correlation between two other variables. ... In the more familiar analysis of variance (ANOVA) terms, a basic moderator effect can be represented as an interaction between a focal independent variable and a factor that specifies the appropriate conditions for its operation." p. 1174

A mediator variable

"In general, a given variable may be said to function as a mediator to the extent that it accounts for the relation between the predictor and the criterion. Mediators explain how external physical events take on internal psychological significance. Whereas moderator variables specify when certain effects will hold, mediators speak to how or why such effects occur." p. 1176

$\endgroup$
  • 4
    $\begingroup$ Could you elaborate on how the distinction between mediators and moderators is relevant to whether a partial correlation can be greater than a zero-order correlation? $\endgroup$ – Jeromy Anglim Apr 25 '11 at 5:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.