What can I infer about a correlation based on another correlation that shares one variable? I have three vectors, $X$, $Y$, and $Z$. Each element of $X$ is independently normally distributed $X_i\sim N(0,\sigma^2_X)$.
The elements of $Y$ and $Z$ are jointly normally distributed:
$\left[ \begin{array}{c}Y_i\\ Z_i\end{array}\right]\sim N\left(\left[\begin{array}{c}0\\ 0\end{array}\right],\left[\begin{array}{cc}\sigma^2_Y & \rho \\ \rho & \sigma^2_Z\end{array}\right]\right)$
So, naturally, $E(X'Y)=E(X'Z)=0$.
However, in finite samples, $X'Y$ doesn't have to be zero.
What can I say about $E(X'Z)$ conditional on my sample observation of $X'Y$? In other words, can I say anything about $E(X'Z|X'Y)$?
I feel like this is probably not a super difficult problem but I've come at it a few different ways and not really been able to make much headway (aside from a simulation, which gives me an answer but not why, or how conditional it is on the parameters I set), which makes me think I'm missing something obvious. 
While in the middle of writing this up it occurred to me that I could probably calculate a distribution of $X'Z$ conditional on $X'Y$ from the Wishart distribution. But I, uh, hope it's simpler than that.
 A: I hope I understand the question, but I think you are mixing things up here. 
Let's start simple. The world has three random variables, $X$, $Y$, and $Z$. You do not know the data generating process. You observe a finite sample of $X$ and $Y$, and can compute their correlation. What can you say about the population level correlation between $X$ and $Z$ (i.e. about the DGP $\sigma_{XZ}$)? Nothing!
Now, assume yo do know the DGP, and it is as in your question. Then the issue is not an issue. Your "finite sample" correlation $\hat \sigma_{XY}$ is in fact non-zero, whereas you know its population level it's zero. But this is not a problem. Actually, it is only by chance (with a probability close to zero) that you will actually get a finite sample value of zero. Furthermore, if you compute your finite sample $\hat \sigma_{YZ}$, this will not be $\rho$ either. What you could do for instance is to bootstrap $\hat \sigma_{XY}$, and if your sample is random, you will find that the latter is no statistically different from zero, just as $\hat \sigma_{YZ}$ will not be statistically different from $\rho$.
