I came across this blog post about least angle regression, and at a point he says:

  • Find the variable $x_1$ most correlated with the residual. (Note that the variable most correlated with the residual is equivalently the one that makes the least angle with the residual, whence the name.)

  • Move in the direction of this variable until some other variable $x_2$ is just as correlated.

But I couldn't visualize it. By "residual" he is referring to the vectors minus the average of the vectors, am I right? What could be a geometrical interpretation of "variable correlation with the residual"? What would be a, let's say, 2D example of this interpretation?

  • 1
    $\begingroup$ stats.stackexchange.com/a/1448 $\endgroup$
    – whuber
    Jun 12, 2012 at 16:40
  • 2
    $\begingroup$ see also "13 ways to look at the correlation coefficient" by Rodgers and Nicewander: data.psych.udel.edu/laurenceau/… $\endgroup$
    – shabbychef
    Jun 12, 2012 at 17:05
  • $\begingroup$ The original LARS paper has several illustrative figures of the procedure. Have you also looked there? The linked blog post has an adapted version of one of them. $\endgroup$
    – cardinal
    Jun 12, 2012 at 17:06
  • $\begingroup$ No pictures, but related: stats.stackexchange.com/questions/6795/… $\endgroup$
    – cardinal
    Jun 12, 2012 at 17:09
  • $\begingroup$ @shabby: this article is incredible! I really love this kind of reasoning, multiple point of views in the same concept... Fantastic! $\endgroup$
    – Lucas Reis
    Jun 13, 2012 at 3:01

1 Answer 1


When two variables are highly correlated it means that when you do a scatter plot of the observed pairs they will fall close to a straight line. It is the same here except that one of the variables is a residual. In two dimensions looking at vectors suppose you move from the point (Xmean, Ymean) to a point (Xmean + D, Ymean + E) That vector will have slope E/D. Now if the residual vector at (Xmean + D, Ymean + E) has a change in X and Y similar to the change for the variable X1 being considered it will be highly correlated with X1.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.