13
$\begingroup$

Does anybody have an idea about why conditional correlation between 2 variables is called "partial" correlation and the simple correlation between them (so, when not conditioned on any other variable) is called "marginal" correlation? What is the intuition behind words "partial" and "marginal"? What do they do with "parts" or "margins"?

It would be good to learn the answer in order to understand those concepts better.

Improve this question
$\endgroup$
1

2 Answers 2

Reset to default
13
$\begingroup$

The term "marginal" is very old. If you go back far enough in history, there were no scientific journals (evidently they started circa 1665). Instead, interim results were communicated via hand-written letters, and final results were written in books. There didn't tend to be much in the way of data graphics before Playfair, but books might often have tables with numbers under different conditions. Consider this table:
\begin{array} \ &A &B &C &D \\ I &x_{I,A} &x_{I,B} &x_{I,C} &x_{I,D} \\ II &x_{II,A} &x_{II,B} &x_{II,C} &x_{II,D} \\ III &x_{III,A} &x_{III,B} &x_{III,C} &x_{III,D} \\ IV &x_{IV,A} &x_{IV,B} &x_{IV,C} &x_{IV,D} \\ \end{array} These values are all conditional; that is, they give a number for a specific combination of conditions. However, sometimes readers wanted to know what a particular condition was like without regard for the other variable. Imagine $x_{I,A}$ is the number of times something happened when the first variable was $I$ and the second variable was $A$. Then, someone might want to know, how often did this happen when the first variable was $I$ no matter what the second variable was? It's easy to figure this out, you just sum up the $x$s in the first row and ignore the columns. People used to do this sort of thing commonly, and they (naturally) wrote the numbers in the margins of the book beside the table. Whereas the original numbers are conditional, there was no name for these other sorts of numbers; they became known as "marginal".

What do those numbers have to do with correlations? Well, it's not a direct connection, but once you have the idea of 'not taking account of other variables', and you have a name for that ("marginal"), when a new context arises that is analogous (i.e., correlations), the name and the idea are simply applied.

I don't know the etymology of partial correlations, but I can give you the intuition. It's rather straightforward, really: you are dealing with the correlation between part of one variable and part of another. Consider this figure:

enter image description here

We can imagine the left circle is a variable $X$, the right circle is a variable $Y$, and the top circle is a variable $Z$. The correlation between two variables is related to how much the circles overlap (in fact, we can imagine the area of the circles represents the variability of each variable and that the percentage of the area is $r^2$). Now it is clear that there is some correlation between $X$ and $Y$, but there is also some correlation between $X$ and $Z$, and between $Y$ and $Z$. What if you wanted to know what the correlation was between those parts of $X$ and $Y$ that were unrelated to $Z$? That would be the partial correlation. It is related to the overlap between the two parts of the circles that don't include the top slivers that intersect with the top circle.

I am fond of this webpage for providing an easy to understand discussion of partial correlations and related topics. Only the first section is about partial correlations per se, but I highly recommend reading the whole page (even though it's rather long). Although not directly related, the discussion at this thread: Where is the shared variance between all IVs in a linear multiple regression equation?, may be helpful as well.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Thanks! Drawing this out leads me to another question regarding symmetry. We know that $\rho(X,Y) = \rho(Y,X)$. Does the same property hold for partial correlation, i.e. would $\rho_{XY|Z} = \rho_{YX|Z}$? Using the formula above, we could write: $\rho_{YX|Z} = \sqrt{ \frac{Area(1)}{Area(X - (2+center))} }$, and I don't think that $\rho_{XY|Z}$ will always be equal to $\rho{YX|Z}$ because the denominators can change (is the size of the circles representing $X$ and $Y$ based on the measure of the set $X$ and the measure of the set $Y$ ?)? $\endgroup$
    – Kiran K.
    Commented Feb 13, 2016 at 16:19
  • 1
    $\begingroup$ That should probably be a new question, @KiranK. It's a good question & we don't want it buried in comments where people won't ever find it. $\endgroup$
    – gung - Reinstate Monica
    Commented Feb 13, 2016 at 16:24
  • $\begingroup$ Good idea, I reposted as a question here: stats.stackexchange.com/questions/195410/… $\endgroup$
    – Kiran K.
    Commented Feb 13, 2016 at 16:34
0
$\begingroup$

The correlation $\rho_{XY}$ between 2 variables $X,Y$ (which you call marginal correlation) indicates that both variables samples show some dependence.

The partial correlation $\rho_{XY \cdot Z}$ measures the residual correlation between $X,Y$ once the influence of the confounding variable $Z$ has been removed by linear regression.

In mathematical terms, this is expressed as: $$\rho_{XY \cdot Z} := \frac{\rho_{XY}-\rho_{XZ}\rho_{YZ}}{\sqrt{1-\rho_{XZ}^2}\sqrt{1-\rho_{YZ}^2}}$$

To illustrate the properties coming from this definition, we can consider two limit cases:

  • if $X$ and $Y$ are both 0% correlated with the variable $Z$, then the partial correlation is the correlation: $$\rho_{XY \cdot Z} = \rho_{XY}$$

  • If $Y$ is 100% correlated with $Z$ however, then the partial correlation is always 0 no matter the value of $\rho_{XY}$.

$$\rho_{XY \cdot Z} = 0$$

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.