I teach an intro statistics class at my university (as a graduate student) and I was scouring the internet for interesting graphs on the history of linear regression when I came upon this picture, presumably from a paper that Pearson once wrote:
I was brainstorming ways to explain to my class why linear regression is called "regression" (for "regression to the mean", as I understand it) and this chart caught my eye, but upon further review I became confused as to what it means.
The website where I got this picture claims that the significance of this chart is that it shows that what makes a line a "regression" line is that its slope is less than 1. OK, I understand that rationale well enough.
But... then what explains the "line of best fit" drawn on the graph? To my eye, it seems like the regression line is the classic least-squares line, and the "line of best fit" is just the line which minimizes the ordinary Euclidean distance to the line from the points (i.e. perpendicular distance to the points).
If that's the difference between the two lines, then I get it. But then is there anything more to read into the difference between the two lines with regards to "regression to the mean?" For example, is there a good intuitive way to explain why using the squared residuals (instead of the Euclidean distance) gives a line which is shallower in slope in this case?