Does it make sense for a partial correlation to be larger than a zero-order correlation? This is probably demonstrating a fundamental lack of understanding of how partial correlations work.
I have 3 variables, x,y,z. When I control for z, the correlation between x and y increases over the correlation between x and y when z was not controlled for.
Does this make sense? I tend to think that when one controls for the effect of a 3rd variable, the correlation should decreases.
Thank you for your help!
 A: Looking at the wikipedia page we have the partial correlation between $X$ and $Y$ given $Z$ is given by:
$$\rho_{XY|Z}=\frac{\rho_{XY}-\rho_{XZ}\rho_{YZ}}{\sqrt{1-\rho_{XZ}^{2}}\sqrt{1-\rho_{YZ}^{2}}}>\rho_{XY}$$
So we simply require
$$\rho_{XY}>\frac{\rho_{XZ}\rho_{YZ}}{1-\sqrt{1-\rho_{XZ}^{2}}\sqrt{1-\rho_{YZ}^{2}}}$$
The right hand side has a global minimum when $\rho_{XZ}=-\rho_{YZ}$.  This global minimum is $-1$.  I think this should explain what's going on.  If the correlation between $Z$ and $Y$ is the opposite sign to the correlation between $Z$ and $X$ (but same magnitude), then the partial correlation between $X$ and $Y$ given $Z$ will always be greater than or equal to the correlation between $X$ and $Y$.  In some sense the "plus" and "minus" conditional correlation tend to cancel out in the unconditional correlation.
UPDATE
I did some mucking around with R, and here is some code to generate a few plots.
partial.plot <- function(r){  
  r.xz<- as.vector(rep(-99:99/100,199))  
  r.yz<- sort(r.xz)  
  r.xy.z <- (r-r.xz*r.yz)/sqrt(1-r.xz^2)/sqrt(1-r.yz^2)  
  tmp2 <- ifelse(abs(r.xy.z)<1,ifelse(abs(r.xy.z)<abs(r),2,1),0)  
  r.all <-cbind(r.xz,r.yz,r.xy.z,tmp2)  
  mycol <- tmp2  
  mycol[mycol==0] <- "red"  
  mycol[mycol==1] <- "blue"  
  mycol[mycol==2] <- "green"  
  plot(r.xz,r.yz,type="n")  
  text(r.all[,1],r.all[,2],labels=r.all[,4],col=mycol)  
}

so you submit partial.plot(0.5) to see when a marginal correlation of 0.5 corresponds to in partial correlation.  The plot is color coded so that red area represents the "impossible" partial correlation, blue area where $|\rho|<|\rho_{XY|Z}|<1$ and the green area where $1>|\rho|>|\rho_{XY|Z}|$  Below is an example for $\rho_{XY}=r=0.5$

A: People are more prone to think of confounders and mediators when it comes to these situations, in the sense that by adjusting for them you're blocking paths, removing indirect associations and therefore getting the direct correlation between the two variables of interest. The issue is that sometimes the variable we are adjusting for is neither a confounder or a mediator: It is a collider. Run the R code below:
N = 1000
X <- rnorm(N)
Y <- rnorm(N)
Z <- X + Y + rnorm(N)

In the code above, we're describing X and Y as independent variables but that, together, cause Z. The respective causal diagram would be the one below:

As the name implies, a collider is a node whose incoming arrows collide with it. This path is blocked. You do not need to adjust for Z to measure the direct correlation between X and Y. But if you adjust, you open the path and you add spurious correlation to your estimate. Let's check this with some R code.
library(ppcor)
cor.test(X,Y)

You will get a correlation of 0.04571798.
pcor.test(X,Y,Z)

You will get a partial correlation of -0.4641393 of X and Y adjusting for Z. We had barely anything, and now we have a reasonable correlation. One may think: Ok, but can we have barely anything and then a strong positive correlation with collider-adjustment? Sure!
N <- 1000
X <- rnorm(N)
Y <- rnorm(N)
Z <- X - Y + rnorm(N)
cor.test(X,Y)
pcor.test(X,Y,Z)

THe correlation between X and Y is -0.01578202 and the partial correlation of X and Y, given Z, is 0.5008563.
We could check what happens with mediators and confounders. Ready?
Let's make Z a confounder variable in respect to the association between X and Y.
N = 1000
Z <- rnorm(N)
X <- Z + rnorm(N)
Y <- Z + rnorm(N)

X <- Z -> Y
cor.test(X,Y)

0.524119
library(ppcor)
pcor.test(X,Y,Z)

-0.02022428
See? You blocked the confounding path, in a way that there is practically no relationship between X and Y (which is expected, based on the way we created such relationships). Now for a mediator:
N = 1000
X <- rnorm(N)
Z <- X + rnorm(N)
Y <- Z + rnorm(N)

X -> Z -> Y
cor.test(X,Y)

0.577676
pcor.test(X,Y,Z)

-0.01984836
The take away message is that in such simple cases of unshielded triples, if we know the causal model, we should not adjust for the collider Z if we want the direct association between X and Y. We should adjust for the confounder Z if we want the direct association of X and Y and we could adjust for the mediator if we want the direct association, but sometimes we want the TOTAL association and therefore we could adjust for the mediator Z or not, depending on what is our goal.
A: I think that variable z in the question is a suppresor variable.
I suggest having a look at: 
Tzelgov, J., & Henik, A. (1991).Suppression situations in psychological research: Definitions, implications, and applications, Psychological Bulletin, 109 (3), 524-536. http://doi.apa.org/psycinfo/1991-20289-001
See also: http://dionysus.psych.wisc.edu/lit/articles/TzelgovJ1991a.pdf
HTH,
dror
A: I think you need to know about moderator and mediator variables. The classic paper is Baron and Kenny [cited 21,659 times]
A moderator variable 

"In general terms, a moderator is a
  qualitative (e.g., sex, race, class)
  or quantitative (e.g., level of
  reward) variable that affects the
  direction and/or strength of the
  relation between an independent or
  predictor variable and a dependent or
  criterion variable. Specifically
  within a correlational analysis
  framework, a moderator is a third
  variable that affects the zero-order
  correlation between two other
  variables. ... In the more familiar
  analysis of variance (ANOVA) terms, a
  basic moderator effect can be
  represented as an interaction between
  a focal independent variable and a
  factor that specifies the appropriate
  conditions for its operation." p.
  1174

A mediator variable 

"In general, a given variable may be
  said to function as a mediator to the
  extent that it accounts for the
  relation between the predictor and the
  criterion. Mediators explain how
  external physical events take on
  internal psychological significance.
  Whereas moderator variables specify
  when certain effects will hold,
  mediators speak to how or why such
  effects occur." p. 1176

