I have three data vectors $A$, $B$ and $C$ that are all more or less correlated.

What I want is a meaningful definition for a measure of the degree of correlation between those parts of $A$ and $B$ that are not correlated with $C$.

I do apologize if this is not fully understandable. I struggle to come to an easier formulation of the problem. However I can give an example:

Example: Say $A$, $B$, $C$ could all be binary, i.e. only contain the values 0 and 1. I then would want to know the number of positions where $A$ and $B$ are 1 and $C$ is 0.

As a side note: How does one "uncorrelate" variables, i.e. given two variables $A$ and $B$ how to split $A$ into a part that is maximally correlated with B and another that is maximally uncorrelated?

  • $\begingroup$ I added the least-squares tag to this question because it provides the answer! $\endgroup$ – whuber Oct 23 '13 at 20:02
  • $\begingroup$ Can you elaborate on this short notice? $\endgroup$ – Trilarion Oct 23 '13 at 21:50

Correlation, according to its usual definition, is the cosine of the angle between vectors. Least squares regression decomposes a vector into a component within the linear span of a given set of vectors and an orthogonal component (the "residuals" or "errors"). Those two components, being at a right angle to each other, have zero correlation. Therefore,

  • A meaningful measure of the "degree of correlation between those parts of $A$ and $B$ that are not correlated with $C$" is the correlation of the residuals of $A$ and $B$ with each other when regressed separately against $C$.

  • The part of $A$ that is maximally correlated with $B$ is the projection of $A$ onto $B$ (the least-squares fit) and the part that is "maximally uncorrelated" is the residual of the regression of $A$ against $B$.

The example in the question suggests that "correlation" may be used there in a less conventional but undefined sense. Note that the residuals of the regressions of $A$ and $B$ on $C$ when all are binary will usually not be binary.

  • $\begingroup$ Thank you for the answer, I think it solves my problem. According to it, I'll now correlate the components of $A$ and $B$ that are both orthogonal to $C$. $\endgroup$ – Trilarion Oct 29 '13 at 13:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.