11
$\begingroup$

I am looking through the linear association between variable $X$ and $Y$ with some controlling variables $\mathbf{Z} =(Z_1, Z_2, Z_3, \dots)$

One approach is to do a regression $E(Y)= \alpha_0 +\alpha_1 X + \sum\beta_iZ_i$ and look at $\alpha_1$.

Another approach is to calculate the partial correlation $\rho_{XY.\mathbf{Z}}$

My first question is which one is more appropriate? (Added: From the comments, they are equivalent but different presentations)

For multivariate normal $(X,Y,\mathbf{Z})$, partial correlation $\rho_{XY . Z}$ would be a better choice as its value can tell us how well $X$ and $Y$ being associated, for example $\rho_{XY . \mathbf{Z}} = \pm 1$ can be interpreted as $X$ and $Y$ have a perfect linear relationship after controlling $\mathbf{Z}$.

How about if some $Z_i$'s are categorical? Is partial correlation still proper to measure the association between $X$ and $Y$ controlling for $Z_i$'s? (Added: It is acceptable by changing categorical variables to dummy and then control for them, just like how we handle them in regression). As when I learnt partial correlation, it was used for multivariate normally distributed variables; I am not sure whether it is still appropriate and meaningful when normality of data is violated (say, highly skewed) or even the continuity is not the case (say, one of our controlling variables is 'place of birth'). The calculation of partial correlation of $X$ and $Y$ controlling $Z$ involve Pearson's correlation $\rho_{XZ}$ and $\rho_{YZ}$ which does not make sense when $Z$ is categorical, which makes $\rho_{XY . Z}$ look weird.

Also, is there any robust version of partial correlation (like kendall's $\tau$/Spearman's rank correlation to Pearson's correlation)?

Raised from ssdecontrol: Regression "works" and "makes sense" with categorical predictors, but correlation is occasionally said to be inappropriate for categorical data. Since regression is partial correlation, we have an apparent paradox.

Thanks.

$\endgroup$
6
  • 1
    $\begingroup$ Partial correlation reflect regression coefficient (stats.stackexchange.com/q/76815/3277), so the choice between them is dictated purely by presentation considerations. Categorical predictors can be incorporated in a linear model by making them dummy (or some other contrast)variables. For your request of "robust" version, this question may be related: stats.stackexchange.com/q/40995/3277. $\endgroup$
    – ttnphns
    Jan 13, 2015 at 11:52
  • 1
    $\begingroup$ Partial correlation and beta coefficient are just different ways to present basically the same thing. Yes, you may use partial correlation with any predictors to be dummy. $\endgroup$
    – ttnphns
    Jan 13, 2015 at 12:06
  • 2
    $\begingroup$ You might take interest in partial Eta squared as the general measure of effect size in linear model (regression, ANOVA). It is SSeffect/(SSeffect+SSerror). For a continuous predictor (a covariate) it equals the square of the partial correlation coefficient. $\endgroup$
    – ttnphns
    Jan 13, 2015 at 12:17
  • 1
    $\begingroup$ @ttnphns i think this question could lead to some insightful answers. Regression "works" and "makes sense" with categorical predictors, but correlation is occasionally said to be inappropriate for categorical data. Since regression is partial correlation, we have an apparent paradox and an explanation is in order $\endgroup$ Jan 13, 2015 at 14:57
  • 1
    $\begingroup$ The only time conclusions based on partial correlation require the variables all to be multivariate normal is if you're testing for conditional independence using the inverse correlation or covariance matrix. If you're making no claims about independence, then a calculation of partial correlation is always legitimate. $\endgroup$
    – goodepic
    Jan 14, 2015 at 7:06

1 Answer 1

1
$\begingroup$

It seems to me that the only unanswered part of your question is the part cited below:

Also, is there any robust version of partial correlation (like kendall's 𝜏 τ /Spearman's rank correlation to Pearson's correlation)?

The same way you can have partial Pearson correlation coefficient, you can have partial Spearman correlation coefficient and also Kendall. See some R code below with the package ppcor that helps you with partial correlation.

library(ppcor)

set.seed(2021)
N <- 1000
X <- rnorm(N)
Y <- rnorm(N)
Z <- rnorm(N)

pcor.test(X, Y, Z, method='pearson')

You will be given an estimate of $-0.01175714$. If you rank the variables, that would be equivalent to the Spearman correlation.

pcor.test(rank(X), rank(Y), rank(Z), method='pearson')

And this way you get a partial spearman correlation of $0.008965395$. But you don't have to do this, you can just changed to spearman in the parameter of the function.

pcor.test(X, Y, Z, method='spearman')

And here we go, $0.008965395$ again. If you want to do the partial Kendall correlation, just changed the method parameter again.

pcor.test(X, Y, Z, method='Kendall')

This time, we got a partial Kendall correlation of $0.006344739$.

If by robust you mean not depending on the distribution of the random variables, among other things, and most importantly, a measure of independence, I recommend you to read about Mutual Information.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.