# Proof of $\operatorname{Corr}\left(X, Y\middle|Z\right) = \operatorname{Corr}(X-\hat X, Y-\hat Y)$

The text I'm reading claims that for multivariate $X$, $Y$, and $Z$, and $\hat X$ and $\hat Y$ are the regression of $X$ and $Y$ on $Z$ we have that

$$\operatorname{Corr}\left(X, Y\middle|Z\right) = \operatorname{Corr}(X-\hat X, Y-\hat Y)$$

However, there was no derivation of this claim. What steps are required to show this identity, and why is the assumption of normal variables required?

• Do you mean $Corr(X|Z, Y|Z)$, or just $Corr(X, Y|Z)$? In the first, you regress Z out of both X and Y; this is the partial correlation between X and Y conditional on Z. In the second, you regress Z out of only Y; this is a semipartial correlation with Y conditional on Z. – Noah Jun 25 '18 at 20:45
• Ah, that's unfortunate notation. I intended $\operatorname{Corr}\left((X,Y)\middle|Z\right)$, which I believe corresponds to $\operatorname{Corr}\left(X|Z,Y|Z\right)$? – Frank Vel Jun 25 '18 at 20:57
• $\operatorname{Corr}(X,Y|Z)$ is perfectly fine notation, see e.g. en.wikipedia.org/wiki/Law_of_total_covariance whereas $\operatorname{Corr}(X|Z,Y|Z)$ is not. – Jarle Tufto Jun 27 '18 at 11:31

This is definitional and comes from the fact that $$X_i = Z_i\beta_x + \epsilon_{xi}$$, $$\hat{X}_i = Z_i\beta_x$$ and $$X_i|Z_i = \epsilon_{xi} = X_i - Z_i\beta_x = X_i - \hat{X}_i$$ (and the same for $Y$). These all come from the definition of regression. Then you can simply substitute: $$Corr(X,Y|Z) = Corr(X|Z, Y|Z) = Corr(X - \hat{X}, Y - \hat{Y})$$.

• So there is no need to assume the variables to be normal? – Frank Vel Jun 25 '18 at 18:59
• Perhaps I'm wrong (I think I received a downvote for this one), but I don't think you need that assumption. – Noah Jun 25 '18 at 20:43
• This doesn't look right to me. Your equation $\hat{X}_i = Z_i \beta_x$ uses the true coefficient $\beta_x$ rather than the corresponding OLS estimator. You seem to be looking at the true error terms rather than residuals. – Ben Jun 26 '18 at 3:11
• That's true, and probably an oversimplification, but I believe this identity works on both the error term and the residuals, i.e., in both the population (using the true $\beta_x$) and the sample (using the estimated $\hat{\beta}_x$). The quantity $X|Z$ doesn't imply that we're in sample mode, and nothing about the structure of the proof necessitates examining either the sample or population. – Noah Jun 26 '18 at 13:58
• @MartijnWeterings In general $\operatorname{Corr}(X,Y|Z)$ depends on the value $Z$ we condition on. So the left hand side don't necessarily equal the right hand side (which is just some constant) except in the multvariate normal case. – Jarle Tufto Jun 27 '18 at 13:00

For a multivariate normal random vector partitioned as $\mathbf{x}=\left[\begin{matrix}\mathbf{x}_1 \\ \mathbf{x}_2\end{matrix}\right]$, the conditional distribution of $\mathbf{x}_1$ conditional on on $\mathbf{x}_2$ is also multivariate normal with mean vector equal to the general expression for the best linear minimum mean square error predictor $\hat{\mathbf{x}}_1$ of $\mathbf{x}_1$ based on $\mathbf{x}_2$ (applying to any multivariate distribution) and, importantly, a variance-covariance matrix that does not depend on $\mathbf{x}_2$.

In the multivariate normal case we can thus write $$\mathbf{x}_1=\hat{\mathbf{x}}_1 + \mathbf{e}_1,$$ where $\mathbf{e}_1$ is independent of $\hat{\mathbf{x}}_1$ and multivariate normal with zero mean and variance-covariance matrix equal to the the above conditional variance-covariance matrix.

Hence, $$\operatorname{Var}(\mathbf{x}_1-\hat{\mathbf{x}}_1) = \operatorname{Var}\mathbf{e}_1=\operatorname{Var}(\mathbf{x}_1|\mathbf{x}_2),$$ that is, all variances, covariances and correlations of deviations from $\hat{\mathbf{x}}_1$ equals the corresponding conditional quantities.

This equality does not hold in general, however, as we can easily construct a multivariate distribution for which the conditional variance-covariance matrix of $\mathbf{x}_1$ depends on $\mathbf{x}_2$.

• I believe I am a bit stuck in definitions. I get that the $x_2$ dependency of $Var(x_1|x_2)$ but would not equally the distribution of the heterogeneous error-term $Var(x_1-\hat{x}_1)$ change depending on $x_2$? Or is this term related to the residuals? Or is the term not considered to be conditional or evaluated over all $x_2$? It is a bit confusing that $\hat{x}_1$ is ambiguous. – Sextus Empiricus Jun 27 '18 at 15:40
• $\hat x_1=\mu_1 + \Sigma_{12}\Sigma_{22}^{-1}(x_2-\mu_2)$ is just another random variable defined on the same sample space as $x_1$ so $\operatorname{Var}(x_1-\hat x_1)$ is just some constant whereas $\operatorname{Var}(x_1|x_2)$ may depend on $x_2$. – Jarle Tufto Jun 27 '18 at 16:19