I have an intuition that correlation tries to find a best fit line and minimizes the distance from points to that line.

I know that regression works by minimizing the squared vertical distance to a best fit line, but this makes regression non-symmetrical (i.e., y~x != x~y), which makes me think that correlation must not be minimizing the distance in the same way.

  • 2
    $\begingroup$ One account that aims at this intuition and explicitly compares correlation to regression is given at stats.stackexchange.com/a/71303/919. Regression does acquire the symmetry you seek when you work in the units of measurement natural to the variables: namely, their standardized values. $\endgroup$
    – whuber
    Feb 3 at 18:21
  • 2
    $\begingroup$ @Jeremy One simple and useful connection between regression and correlation (sadly, frequently left unexplored by elementary books) is that that the fitted slope $b$ is the one that makes the correlation between the residuals and $x$ zero, $\text{cor}(y-bx,x)=0$. That is, you choose as your slope estimate the one that leaves no residual correlation. If you use Pearson correlation the fitted slope $b$ is the usual least squares simple regression slope. When you measure correlation with Kendall's tau you get the Theil-Sen slope ...ctd $\endgroup$
    – Glen_b
    Feb 3 at 23:25
  • $\begingroup$ ... and so on (not all such slope estimates have common names, though). See stats.stackexchange.com/a/110112/805 for an example that considers several correlation measures. While likely not what you were after, it is a connection worth knowing about. $\endgroup$
    – Glen_b
    Feb 4 at 8:00


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.