I was just wondering why regression problems are called "regression" problems. What is the story behind the name?
One definition for regression: "Relapse to a less perfect or developed state."
The term "regression" was used by Francis Galton in his 1886 paper "Regression towards mediocrity in hereditary stature". To my knowledge he only used the term in the context of regression toward the mean. The term was then adopted by others to get more or less the meaning it has today as a general statistical method.
As opposed to progressing, we are falling back to the mean, i.e. regressing. Hence the term regression ! I think its something that got picked up and stuck.
@Mark White mentioned the link already but for those of you who do not have much time to check the link, here's the exact properly referenced answer:
The term "regression" was coined by Francis Galton in the 19th century to describe a biological phenomenon. The phenomenon was that the heights of descendants of tall ancestors tend to regress down towards a normal average (a phenomenon also known as regression toward the mean)(Galton, reprinted 1989). For Galton, regression had only this biological meaning (Galton, 1887), but his work was later extended by Udny Yule and Karl Pearson to a more general statistical context (Pearson, 1903).
Galton, F. (1877). Typical laws of heredity. III. Nature, 15(389), 512-514.
Galton, F. (reprinted 1989). Kinship and Correlation. Statistical Science, 4(2), 80–86.
Pearson, K. (1903). The law of ancestral heredity. Biometrika, 2(2), 211-228.
"Regression" comes from "regress" which in turn comes from latin "regressus" - to go back (to something).
In that sense, regression is the technique that allows "to go back" from messy, hard to interpret data, to a clearer and more meaningful model. As a physicist, I like the idea, as physicists see natural phenomena as the multiple possible outcomes of a relatively simple natural law.
In other words, the word regression seems to suggest that data is just the visible, tangible effect of a "statistical model". In other words, the model comes first, and your desire is use the data "to go back" to what originated them.
As I know the word of
regression in statistical meaning is the measurement of the relation between the mean value of one variable and the corresponding values of other variables.