19
$\begingroup$

This is probably demonstrating a fundamental lack of understanding of how partial correlations work.

I have 3 variables, x,y,z. When I control for z, the correlation between x and y increases over the correlation between x and y when z was not controlled for.

Does this make sense? I tend to think that when one controls for the effect of a 3rd variable, the correlation should decreases.

Thank you for your help!

Improve this question
$\endgroup$
1
  • $\begingroup$ I can't top what probabilityislogic has done, but for a light treatment that gives illustrative examples and requires no math, see integrativestatistics.com/partial.htm $\endgroup$
    – rolando2
    Commented Apr 27, 2011 at 1:35

4 Answers 4

Reset to default
8
$\begingroup$

Looking at the wikipedia page we have the partial correlation between $X$ and $Y$ given $Z$ is given by:

$$\rho_{XY|Z}=\frac{\rho_{XY}-\rho_{XZ}\rho_{YZ}}{\sqrt{1-\rho_{XZ}^{2}}\sqrt{1-\rho_{YZ}^{2}}}>\rho_{XY}$$

So we simply require

$$\rho_{XY}>\frac{\rho_{XZ}\rho_{YZ}}{1-\sqrt{1-\rho_{XZ}^{2}}\sqrt{1-\rho_{YZ}^{2}}}$$

The right hand side has a global minimum when $\rho_{XZ}=-\rho_{YZ}$. This global minimum is $-1$. I think this should explain what's going on. If the correlation between $Z$ and $Y$ is the opposite sign to the correlation between $Z$ and $X$ (but same magnitude), then the partial correlation between $X$ and $Y$ given $Z$ will always be greater than or equal to the correlation between $X$ and $Y$. In some sense the "plus" and "minus" conditional correlation tend to cancel out in the unconditional correlation.

UPDATE

I did some mucking around with R, and here is some code to generate a few plots.

partial.plot <- function(r){  
  r.xz<- as.vector(rep(-99:99/100,199))  
  r.yz<- sort(r.xz)  
  r.xy.z <- (r-r.xz*r.yz)/sqrt(1-r.xz^2)/sqrt(1-r.yz^2)  
  tmp2 <- ifelse(abs(r.xy.z)<1,ifelse(abs(r.xy.z)<abs(r),2,1),0)  
  r.all <-cbind(r.xz,r.yz,r.xy.z,tmp2)  
  mycol <- tmp2  
  mycol[mycol==0] <- "red"  
  mycol[mycol==1] <- "blue"  
  mycol[mycol==2] <- "green"  
  plot(r.xz,r.yz,type="n")  
  text(r.all[,1],r.all[,2],labels=r.all[,4],col=mycol)  
}

so you submit partial.plot(0.5) to see when a marginal correlation of 0.5 corresponds to in partial correlation. The plot is color coded so that red area represents the "impossible" partial correlation, blue area where $|\rho|<|\rho_{XY|Z}|<1$ and the green area where $1>|\rho|>|\rho_{XY|Z}|$ Below is an example for $\rho_{XY}=r=0.5$

Partial correlation when marginal correlation is 0.5

Improve this answer
$\endgroup$
4
  • $\begingroup$ The code does not work for me. Line 5 looks strange and Emacs tells me that some parentheses are missing. $\endgroup$
    – Bernd Weiss
    Commented Apr 26, 2011 at 10:50
  • $\begingroup$ There's a missing "return" - must have accidentally deleted it. should be good now. $\endgroup$
    – probabilityislogic
    Commented Apr 26, 2011 at 11:00
  • $\begingroup$ Thanks, works now! However, I am having some difficulties to understand the plot: is it really $r_{xz}$ on both axes? $\endgroup$
    – Bernd Weiss
    Commented Apr 26, 2011 at 11:31
  • $\begingroup$ dog gone it! need to fix my code... again - plot() is wrong. Ah the glorious tedium of computer code $\endgroup$
    – probabilityislogic
    Commented Apr 26, 2011 at 12:19
7
$\begingroup$

People are more prone to think of confounders and mediators when it comes to these situations, in the sense that by adjusting for them you're blocking paths, removing indirect associations and therefore getting the direct correlation between the two variables of interest. The issue is that sometimes the variable we are adjusting for is neither a confounder or a mediator: It is a collider. Run the R code below:

N = 1000
X <- rnorm(N)
Y <- rnorm(N)
Z <- X + Y + rnorm(N)

In the code above, we're describing X and Y as independent variables but that, together, cause Z. The respective causal diagram would be the one below:

enter image description here

As the name implies, a collider is a node whose incoming arrows collide with it. This path is blocked. You do not need to adjust for Z to measure the direct correlation between X and Y. But if you adjust, you open the path and you add spurious correlation to your estimate. Let's check this with some R code.

library(ppcor)
cor.test(X,Y)

You will get a correlation of 0.04571798.

pcor.test(X,Y,Z)

You will get a partial correlation of -0.4641393 of X and Y adjusting for Z. We had barely anything, and now we have a reasonable correlation. One may think: Ok, but can we have barely anything and then a strong positive correlation with collider-adjustment? Sure!

N <- 1000
X <- rnorm(N)
Y <- rnorm(N)
Z <- X - Y + rnorm(N)
cor.test(X,Y)
pcor.test(X,Y,Z)

THe correlation between X and Y is -0.01578202 and the partial correlation of X and Y, given Z, is 0.5008563.

We could check what happens with mediators and confounders. Ready?

Let's make Z a confounder variable in respect to the association between X and Y.

N = 1000
Z <- rnorm(N)
X <- Z + rnorm(N)
Y <- Z + rnorm(N)

X <- Z -> Y

cor.test(X,Y)

0.524119

library(ppcor)
pcor.test(X,Y,Z)

-0.02022428

See? You blocked the confounding path, in a way that there is practically no relationship between X and Y (which is expected, based on the way we created such relationships). Now for a mediator:

N = 1000
X <- rnorm(N)
Z <- X + rnorm(N)
Y <- Z + rnorm(N)

X -> Z -> Y

cor.test(X,Y)

0.577676

pcor.test(X,Y,Z)

-0.01984836

The take away message is that in such simple cases of unshielded triples, if we know the causal model, we should not adjust for the collider Z if we want the direct association between X and Y. We should adjust for the confounder Z if we want the direct association of X and Y and we could adjust for the mediator if we want the direct association, but sometimes we want the TOTAL association and therefore we could adjust for the mediator Z or not, depending on what is our goal.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ Very nice connection with graphical models ! $\endgroup$
    – Thomas
    Commented Dec 10, 2021 at 12:15
  • 1
    $\begingroup$ Instead, if you have a confounder variable, you have a configuration where one nodes have the same cause (are sons of the same father). In such a case conditioning on the confounder variable should always descrease correlations right? It would be nice to "merge" your answer with the one of @probabilityislogic $\endgroup$
    – Thomas
    Commented Dec 10, 2021 at 12:20
  • 1
    $\begingroup$ By the way, what is a "mediator" variable? What would be the corresponding graph topology? Maybe just a chain? $\endgroup$
    – Thomas
    Commented Dec 10, 2021 at 12:21
  • 1
    $\begingroup$ Regarding your first comment, @Thomas, yes. The amount of association between X and Y occurs indirectly through the confounding variable and directly without it, so the total correlation is the sum of both. This way, by adjusting for the confounder (blocking the path) you can not have more than the sum of both effects. For the second question, yes, just a chain. A causes B that causes C. If the only association between A and C happens through B, adjusting for B blocks the path and therefore make A and C independent. Sometimes, however, you have both A causing C directly and through B. $\endgroup$
    – mribeirodantas
    Commented Dec 10, 2021 at 13:07
3
$\begingroup$

I think that variable z in the question is a suppresor variable.

I suggest having a look at:

Tzelgov, J., & Henik, A. (1991).Suppression situations in psychological research: Definitions, implications, and applications, Psychological Bulletin, 109 (3), 524-536. http://doi.apa.org/psycinfo/1991-20289-001

See also: http://dionysus.psych.wisc.edu/lit/articles/TzelgovJ1991a.pdf

HTH, dror

Improve this answer
$\endgroup$
2
$\begingroup$

I think you need to know about moderator and mediator variables. The classic paper is Baron and Kenny [cited 21,659 times]

A moderator variable

"In general terms, a moderator is a qualitative (e.g., sex, race, class) or quantitative (e.g., level of reward) variable that affects the direction and/or strength of the relation between an independent or predictor variable and a dependent or criterion variable. Specifically within a correlational analysis framework, a moderator is a third variable that affects the zero-order correlation between two other variables. ... In the more familiar analysis of variance (ANOVA) terms, a basic moderator effect can be represented as an interaction between a focal independent variable and a factor that specifies the appropriate conditions for its operation." p. 1174

A mediator variable

"In general, a given variable may be said to function as a mediator to the extent that it accounts for the relation between the predictor and the criterion. Mediators explain how external physical events take on internal psychological significance. Whereas moderator variables specify when certain effects will hold, mediators speak to how or why such effects occur." p. 1176

Improve this answer
$\endgroup$
1
  • 4
    $\begingroup$ Could you elaborate on how the distinction between mediators and moderators is relevant to whether a partial correlation can be greater than a zero-order correlation? $\endgroup$
    – Jeromy Anglim
    Commented Apr 25, 2011 at 5:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.