6
$\begingroup$

There are a lot of geometric intuitions for regression with least square, e.g., projection, orthogonal, etc. (This and this answers are good examples.)

Is there similar geometric intuition for least absolute deviation regression?

$\endgroup$
  • 3
    $\begingroup$ The technique, which goes back at least to 1757, antedates all those other methods. See the figure on p. 48 of Stigler's The History of Statistics and the preceding explanation in the text. $\endgroup$ – whuber Jan 23 '18 at 19:21
  • 2
    $\begingroup$ I have read that book several times, it's on my shelf, and it was the first reference that came to mind when I saw your question. I think I have some other explanations around, too, from the publications by the Princeton-Harvard groups in the 1980's on EDA and robust methods, but I would have to search harder to find the relevant material. $\endgroup$ – whuber Jan 23 '18 at 19:52
  • 1
    $\begingroup$ @whuber is it true (I seem to recall hearing) people generally believe(d) (then and now) that LAD visually matches what the eye sees as the line of best fit to the data whereas least squares usually appears somewhat flat? $\endgroup$ – AdamO Jan 24 '18 at 16:04
  • 3
    $\begingroup$ @AdamO I hadn't heard that before. There is a body of literature, such as Bill Cleveland's studies from the 1980's, that indicates people tend to respond to angles rather than vertical deviations (as I recall--I'm away from my usual library today). If this is correct, then the best "visual match" might be the first principal component (the major ellipse of the point cloud). It is plausible that often LAD might lie closer to that axis than the OLS line, which exhibits regression to the mean. Whether that's a useful or desirable property is a different matter! $\endgroup$ – whuber Jan 24 '18 at 16:18
  • 1
    $\begingroup$ Possible duplicate of How does quantile regression "work"? $\endgroup$ – kjetil b halvorsen Jul 31 '18 at 11:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.