Relationship between correlation coefficient and the angle between two regression lines

Question

From: https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

Geometric interpretation Regression lines for $y = g_X(x)$ [red] and $x = g_Y(y)$ [blue] Regression lines for $y = g_X(x)$ [red] and $x = g_Y(y)$ [blue]

For uncentered data, there is a relation between the correlation coefficient and the angle $\varphi$ between the two regression lines, $y = g_X(x)$ and $x = gY(y)$, obtained by regressing $y$ on $x$ and $x$ on $y$ respectively. (Here, $\varphi$ is measured counterclockwise within the first quadrant formed around the lines' intersection point if $r > 0$, or counterclockwise from the fourth to the second quadrant if $r < 0$.) One can show[18] that if the standard deviations are equal, then $r = \sec \varphi$ − tan $\varphi$, where sec and tan are trigonometric functions.

How can we prove this relationship mentioned above: $r = \sec \varphi − \tan \varphi$?

Answer 1

Working based on the listed reference ([18]).

The equations of the two regression lines are

$$ x= a_{xy} + b_{xy} y \\ y= a_{yx} + b_{yx} x $$

If we graph these, then they will have gradients of $b_{yx}$ and $\frac{1}{b_{xy}}$, respectively.

The relationship between these regression coefficients, the standard deviations ${s_x}, {s_y}$ on $x$ and $y$, and the correlation $r$ is:

$$ b_{xy} = r\frac{s_x}{s_y},\quad \text{and} \quad b_{yx} = r\frac{s_y}{s_x} $$

So, in the case that $s_x=s_y$, the two regression lines have gradients of $r$ and $1/r$.

In the following, I will assume that $0<r<1$. In the case that $r=1$, the two lines are the same line, and so the angle is zero; and in the case that $r=0$, the two lines are orthogonal. You can show that each of these cases satisfy the identity that you care about with substitution. For the case that $r<0$, the same idea of the proof works, you just need to flip the entire picture upside down in your mind.

Lemma 1: the angle between two lines with gradients $m_1$ and $m_2$ is given by the equation

$$\tan (\phi) = \frac{m_1 - m_2}{1+m_1m_2}$$

Proof: The line with gradient $m_1$ makes an angle of $\theta_1 = \tan^{-1}(m_1)$ with the $x$-axis. The line with gradient $m_2$ makes an angle of $\theta_2 = \tan^{-1}(m_2)$ with the $x$-axis. So the angle between the lines is:

$$ \phi = \theta_1 - \theta_2 = \tan^{-1}(m_1) - \tan^{-1}(m_2) $$

Then we can use the compound angle formula for tan:

$$ \tan(\phi) = \tan\left( \tan^{-1}(m_1) - \tan^{-1}(m_2) \right) \\ =\frac{m_1-m_2}{1+m_1m_2} \qquad \square $$

--

So, applying this to our equation, we learn that

$$ \tan(\phi) = \frac{1}{2}\left(\frac{1}{r} - r\right) $$

We can show that this equation is satisfied by $r=\sec(\phi)-\tan(\phi)$ by substituting this equation into the RHS, and showing that it reduces to the LHS.

Let $r=\frac{1-\sin(\phi)}{\cos{\phi}}$. Then:

$$ RHS = \frac{1}{2}\left(\frac{1}{r} - r\right) = \frac{1}{2}\left(\frac{\cos{\phi}}{1-\sin(\phi)} - \frac{1-\sin(\phi)}{\cos{\phi}}\right) \\ = \frac{1}{2}\left(\frac{ \cos (\phi)^2 - (1-\sin(\phi))^2}{\cos(\phi)(1-\sin(\phi))}\right) \\ = \frac{1}{2}\left(\frac{ 1 - \sin(\phi)^2 - \left(1-2\sin(\phi)+\sin(\phi)^2\right)}{\cos(\phi)(1-\sin(\phi))}\right) \\ = \frac{1}{2}\left(\frac{ 2\sin(\phi) - 2\sin(\phi)^2}{\cos(\phi)(1-\sin(\phi))}\right) \\ = \frac{\sin(\phi)}{\cos(\phi)} = LHS \quad \square $$