Let $X,Y$ be real random variables with finite variances, and with no loss of generality assume $\mathbf{E}[X] = \mathbf{E}[Y]=0$. For simplicity, I will focus on the case $\mathrm{Var}X \neq \mathrm{Var} Y$.

Part 1: OLS

Naïvely, we could perform OLS by doing $Y = \beta X$ and obtain the standard $\beta_{OLS} = \rho \sigma_Y/\sigma_X$. This means that the angle the regression line makes with the $x$-axis is $\tan\theta_{OLS} = \rho \sigma_Y/\sigma_X$, and that the double of that angle would be (I'll need this result later) $$\tan(2\theta_{OLS}) = \frac{2\tan\theta_{OLS}}{1-\tan^2\theta_{OLS}} = \frac{2\rho \sigma_X \sigma_Y}{\sigma_X^2 -\color{red}{\rho^2} \sigma_Y^2} \qquad (*)$$

Part 2: rotation

In a different direction, we can calculate the angle $\theta$ such that the variables $X', Y'$, given by, $$\begin{pmatrix} X' \\ Y'\end{pmatrix} = \begin{pmatrix} \cos\theta & \sin\theta \\ - \sin\theta & \cos \theta \end{pmatrix} \begin{pmatrix} X \\ Y \end{pmatrix}$$ are uncorrelated. By simply expanding $\mathbf{E}[X'Y'] = 0$ we get $$\tan(2\theta) = \frac{2\rho \sigma_X \sigma_Y}{\sigma_X^2-\sigma_Y^2}\qquad (**)$$

Now, (*) and (**) are different. My questions are then: 1. Should they be the same? I.e. best linear regression providing the angle under which data loses correlation if rotated; 2. In case they should be the same, could the reason for the difference be thate OLS only considers errors in $Y$, not $X$? I would like to try a errors-in-variables model so that a modified version of (*) would work.

Thank you very much!

  • 1
    $\begingroup$ The answers to questions (1) and (2) are "no" and "yes" respectively, as you have clearly shown. $\endgroup$ – whuber Sep 11 '17 at 15:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.