0
$\begingroup$

I have 10 different 2D datasets which each have 200 datapoints. Each data point has a 'quality' property ($Q$) which somewhat depends on the area ($A$) of that datapoint. There is a (relatively) strong linear relationship between these two properties for most of the datasets (row 1 of the figure). Call the correlation coefficients here $r_1$.

Data showing relationship between correlation coefficients

I then calculate the residuals ($R = Q - \hat Q$) of the linear relationship between the quality and the area (row 2). There is still a relatively strong relationship between the residual and the quality (row 3). Call the correlation coefficients here $r_2$. [Subquestion: what does this mean about the relationship between quality and area?]

If I take the Pearson correlation coefficient of these two linear relationships, these very strongly follow the pattern $r_1^2 + r_2^2 = 1$. My main question: Is this mathematically expected?

enter image description here

I know $ r_1 = \frac{cov(Q, A)}{\sigma_Q \sigma_A}$ and $ r_2 = \frac{cov(Q, R)}{\sigma_Q \sigma_R}$ but I am not able to make any significant inroads as to a proof of the above equality. I have also tried experimenting with some random data, and it seems as though this relationship does not hold in general. So my other question is, what features of the data set will result in this property holding?

$\endgroup$
1
  • $\begingroup$ Re "still a relatively strong relationship:" not if you're using ordinary least squares regression with an intercept term. The correlation between the residuals and any explanatory variable, like Area, is necessarily zero. Please explain what you mean by "the Pearson correlation coefficient of these two linear relationships:" it's not at all clear what that means, because correlation coefficients are defined for data or bivariate distributions but not for "linear relationships." $\endgroup$
    – whuber
    Commented May 10, 2022 at 15:47

1 Answer 1

1
$\begingroup$

I ended up finding the proof using properties of the covariance.

Claim: Given two variables $A$ and $B$ and the linear regression between them $B = mA + c + R$ (where $R$ are the residuals and thus $cov(A, R) = 0$), $\rho_{A,B}^2 + \rho_{B,R}^2 = 1$.

====

Proof: By definition, $\displaystyle \rho_{X,Y} = \frac{cov(X, Y)}{\sigma_X \sigma_Y}$.

Then,

\begin{eqnarray*} \rho_{A,B}^2 + \rho_{B,R}^2 &=& \frac{cov^2(A, B)}{\sigma^2_A \sigma^2_B} + \frac{cov^2(B, R)}{\sigma^2_B \sigma^2_R} \\ &=& \frac{cov^2(A, mA + c + R)}{\sigma^2_A \sigma^2_B} + \frac{cov^2(R, mA + c + R)}{\sigma^2_R \sigma^2_B} \\ &=& \frac{[mcov(A,A) + cov(A,R)]^2}{\sigma^2_A \sigma^2_B} + \frac{[mcov(A,R) + cov(R,R)]^2}{\sigma^2_R \sigma^2_B} \\ &=& \frac{[m \sigma^2_A]^2}{\sigma^2_A \sigma^2_B} + \frac{[\sigma^2_R]^2}{\sigma^2_R \sigma^2_B} \\ &=& \frac{m^2 \sigma^2_A}{\sigma^2_B} + \frac{\sigma^2_R}{\sigma^2_B} \\ &=& \frac{m^2 \sigma^2_A + \sigma^2_R}{cov(mA+c+R, mA+c+R)}\\ &=& \frac{m^2 \sigma^2_A + \sigma^2_R}{m^2 cov(A,A) + 2mcov(A,R) + cov(R,R)}\\ &=& \frac{m^2 \sigma^2_A + \sigma^2_R}{m^2 \sigma^2_A + \sigma^2_R}\\ &=& 1 \end{eqnarray*}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.