2
$\begingroup$

Here (page 415) https://www.stat.cmu.edu/~larry/=sml/DAGs.pdf I found this definition:

enter image description here

which confuses me. I am used to see $E[*|Z]$ as a ($Z$-measurable) random variable and as far as I know the partial correlation is a deterministic scalar, as I understand from the Wikipedia article from example:

https://en.wikipedia.org/wiki/Partial_correlation

What is the meaning intended by the author ? Is this definition intended to be the same used by Wikipedia? (the text in the definition would suggest so...)

( in order to simplify things, let's consider X, Y and Z scalars for the moment )

$\endgroup$
14
  • $\begingroup$ If $\rho_{X,Y\mid Z=z}$ is meaningful (possibly changing with $z$, i.e. a function of $z$) then $\rho_{X,Y\mid Z}$ is also meaningful (a function of $Z$). $\endgroup$
    – Henry
    Dec 8, 2021 at 23:25
  • $\begingroup$ Correlation does not have to be a scaler. There are correlation matrices if $X$ and $Y$ are vectors. But this is unaffected by whether $Z$ is a scalar or not $\endgroup$
    – Henry
    Dec 8, 2021 at 23:28
  • $\begingroup$ Thanks @Henry . I updated my question. In the definition of Wikipedia for my understanding the partial correlation is a deterministic scalar if X, Y and Z are scalars r.v.. Let's consider this simple case for the moment. But E[X|Z] in this case is a scalar r.v. and not a deterministic scalar for my notation... I cannot reconcile these definitions at the moment... $\endgroup$
    – Thomas
    Dec 9, 2021 at 0:18
  • $\begingroup$ $E[X\mid Z]$ is presumably a function of $Z$. So if $Z$ is a random variable, then so too is $E[X\mid Z]$ $\endgroup$
    – Henry
    Dec 9, 2021 at 0:42
  • 1
    $\begingroup$ @ttnphns The definition above in the original post is in the context of jointly normal $(X,Y,Z)$ and not in general (as I found in the linked document). In this special case, these are all formal conditional expectations and variances and the 'conditional correlation' between $X$ and $Y$, given $Z$, equals the usual partial correlation. This is what I understood anyway. $\endgroup$ Dec 12, 2021 at 10:01

1 Answer 1

1
$\begingroup$

Suppose you have a random vector $\boldsymbol X=(X_1,X_2,\ldots,X_p)$.

Consider the linear regression models

$$X_1=X_{1\cdot 34\ldots p}+ \varepsilon_{1\cdot 34\ldots p}$$

and

$$X_2=X_{2\cdot 34\ldots p}+ \varepsilon_{2\cdot 34\ldots p}$$

Here $X_{i\cdot 34\ldots p}$ is the part of $X_i$ explained by $(X_3,\ldots,X_p)$ and $\varepsilon_{i\cdot 34\ldots p}$ is the unexplained error, $i=1,2$. Unknown parameters in $X_{i\cdot 34\ldots p}$ are found subject to minimization of $E(\varepsilon_{i\cdot 34\ldots p}^2)$.

If $e_{i.34\ldots p}$ are the residuals corresponding to the models above, the (population) partial correlation between $X_1$ and $X_2$, eliminating the linear effect of $X_3,\ldots,X_p$, is defined to be

$$\rho_{12\cdot 34\ldots p}=\operatorname{Corr}(e_{1.34\ldots p},e_{2.34\ldots p}) $$

For some distributions like the multivariate normal, this correlation coincides with the correlation between $X_1$ and $X_2$, conditioned on $X_3,\ldots,X_p$. In fact, your linked document does assume multivariate normality of $\boldsymbol X$. To quote Wikipedia, "The partial correlation coincides with the conditional correlation if the random variables are jointly distributed as the multivariate normal, other elliptical, multivariate hypergeometric, multivariate negative hypergeometric, multinomial or Dirichlet distribution, but not in general otherwise."

Hence in these specific situations only, one can say

$$ \rho_{12\cdot 34\ldots p}=\rho_{(X_1,X_2)\mid X_3,\ldots,X_p}=\operatorname{Corr}((X_1,X_2) \mid X_3,\ldots,X_p) \tag{$\star$} $$

When $\boldsymbol X$ is multivariate normal, this conditional correlation (i.e. the conditional covariance and the conditional variances) does not depend on $X_3,\ldots,X_p$. Note that the conditional distribution of $(X_1,X_2)$ given $X_3,\ldots,X_p$ is bivariate normal. And you can see here that the dispersion matrix of this conditional distribution is free of $X_3,\ldots,X_p$ (hence non-random). Hence in this case, there is no ambiguity in the formula in your post.

The partial correlation is of course a scalar by definition. In fact, it is entirely based on the entries of the correlation matrix (or equivalently, the dispersion matrix) of $\boldsymbol X$.

Specifically, if $R=((\rho_{ij}))$ is the correlation matrix of $\boldsymbol X$, one can show that

$$\rho_{12\cdot 34\ldots p}=-\frac{R_{12}}{\sqrt{R_{11}}\sqrt{R_{22}}}\,,$$

where $R_{ij}$ is the cofactor of $\rho_{ij}$.

For $p=3$ (say), this reduces to

$$\rho_{12\cdot 3}=\frac{\rho_{12}-\rho_{13}\rho_{23}}{\sqrt{1-\rho_{13}^2}\sqrt{1-\rho_{23}^2}}$$

Related: Derivation of the formula for partial correlation coefficient of second order.

$\endgroup$
7
  • $\begingroup$ Thank you (+1). You say: "This is also sometimes called the correlation between X1 and X2, keeping X3,…,Xp 'fixed'". Isn't it better said that it is the correlation between X1 and X2 when subtracting from both X1 and X2 the part that can be predicted from "X3,…,Xp" ? What sense does it have to say that "X3,…,Xp" are "fixed" ? (they vary no?) $\endgroup$
    – Thomas
    Dec 9, 2021 at 18:18
  • $\begingroup$ I am not following here: "I can see (⋆) as a formal notation only if the entries of the dispersion matrix of the conditional distribution of (X1,X2) given X3,…,Xp are constants." "constants" with respect to what? You mean when the conditional distribution of (X1,X2) does not depend on the values of "X3,…,Xp"? (i.e. when (X1,X2) is independent of "X3,…,Xp"?) $\endgroup$
    – Thomas
    Dec 9, 2021 at 18:21
  • $\begingroup$ I only mean when the conditional variances and conditional covariance (entries of the conditional dispersion matrix) do not depend on $X_3,\ldots,X_p$. Regarding your first comment, you are right and by 'fixed' I mean we are conditioning on them. $\endgroup$ Dec 9, 2021 at 19:30
  • $\begingroup$ Ok so this would mean that in these cases $E[XY|Z]-E[X|Z]E[Y|Z]=E[(X-E[X|Z])(Y-E[Y|Z])|Z]$ does not depend on Z in conclusion? This is interesting (or should it be trivial?), and is exactly the numerator of the formula of my original post. But at least Var(X|Z) (in the denominator) will depend on Z also in the Gaussian case, or not? $\endgroup$
    – Thomas
    Dec 9, 2021 at 20:14
  • $\begingroup$ Ah no maybe what does not depend on Z will be all the original formula with denominators included (so the correlation, not the covariance), as you wrote in your post... $\endgroup$
    – Thomas
    Dec 9, 2021 at 20:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.