7
$\begingroup$

There are a lot of geometric intuitions for regression with least square, e.g., projection, orthogonal, etc. (This and this answers are good examples.)

Is there similar geometric intuition for least absolute deviation regression?

$\endgroup$
8
  • 3
    $\begingroup$ The technique, which goes back at least to 1757, antedates all those other methods. See the figure on p. 48 of Stigler's The History of Statistics and the preceding explanation in the text. $\endgroup$
    – whuber
    Jan 23, 2018 at 19:21
  • 3
    $\begingroup$ I have read that book several times, it's on my shelf, and it was the first reference that came to mind when I saw your question. I think I have some other explanations around, too, from the publications by the Princeton-Harvard groups in the 1980's on EDA and robust methods, but I would have to search harder to find the relevant material. $\endgroup$
    – whuber
    Jan 23, 2018 at 19:52
  • 1
    $\begingroup$ @whuber is it true (I seem to recall hearing) people generally believe(d) (then and now) that LAD visually matches what the eye sees as the line of best fit to the data whereas least squares usually appears somewhat flat? $\endgroup$
    – AdamO
    Jan 24, 2018 at 16:04
  • 3
    $\begingroup$ @AdamO I hadn't heard that before. There is a body of literature, such as Bill Cleveland's studies from the 1980's, that indicates people tend to respond to angles rather than vertical deviations (as I recall--I'm away from my usual library today). If this is correct, then the best "visual match" might be the first principal component (the major ellipse of the point cloud). It is plausible that often LAD might lie closer to that axis than the OLS line, which exhibits regression to the mean. Whether that's a useful or desirable property is a different matter! $\endgroup$
    – whuber
    Jan 24, 2018 at 16:18
  • 1
    $\begingroup$ Possible duplicate of How does quantile regression "work"? $\endgroup$ Jul 31, 2018 at 11:26

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.