41
$\begingroup$

I was just wondering why regression problems are called "regression" problems. What is the story behind the name?

One definition for regression: "Relapse to a less perfect or developed state."

$\endgroup$
1
36
$\begingroup$

The term "regression" was used by Francis Galton in his 1886 paper "Regression towards mediocrity in hereditary stature". To my knowledge he only used the term in the context of regression toward the mean. The term was then adopted by others to get more or less the meaning it has today as a general statistical method.

$\endgroup$
4
  • 16
    $\begingroup$ Galton derived a linear approximation to estimate a son's height from the father's height in that paper. His equation was fitted so an average height father would have an average height son, but a taller than average father would have a son that is taller than average by 2/3 the amount his father is. Same with shorter than average. This could be argued to be a simple linear regression (today's meaning). And of course today regression has an even broader meaning: it's any model that makes continuous predictions. It is interesting how much his original usage of that word has changed. $\endgroup$ – rm999 May 21 '11 at 19:07
  • 3
    $\begingroup$ Answer by NRH is correct. The following link gives lot more details on Francis Galton's paper "Regression towards mediocrity in hereditary stature" blog.minitab.com/blog/statistics-and-quality-data-analysis/… $\endgroup$ – Gaurav Singhal May 9 '16 at 8:23
  • $\begingroup$ is it time for the statistics community to replace the word 'regression' with a more straightforward and clear term, maybe 'formulaic predictor' ? $\endgroup$ – Aviad Rozenhek Oct 27 '19 at 9:36
  • $\begingroup$ Galton used data from both parents. Resist paraphrases and mangled memes: the original paper is at galton.org/essays/1880-1889/… $\endgroup$ – Nick Cox Aug 2 '20 at 11:21
6
$\begingroup$

@Mark White mentioned the link already but for those of you who do not have much time to check the link, here's the exact properly referenced answer:

Origin of 'regression'

The term "regression" was coined by Francis Galton in the 19th century to describe a biological phenomenon. The phenomenon was that the heights of descendants of tall ancestors tend to regress down towards a normal average (a phenomenon also known as regression toward the mean)(Galton, reprinted 1989). For Galton, regression had only this biological meaning (Galton, 1887), but his work was later extended by Udny Yule and Karl Pearson to a more general statistical context (Pearson, 1903).

References

https://en.wikipedia.org/wiki/Regression_analysis#History

Galton, F. (1877). Typical laws of heredity. III. Nature, 15(389), 512-514.

Galton, F. (reprinted 1989). Kinship and Correlation. Statistical Science, 4(2), 80–86.

Pearson, K. (1903). The law of ancestral heredity. Biometrika, 2(2), 211-228.

$\endgroup$
2
  • $\begingroup$ Galton's regression as in 'regression to the mean' makes sense. however I don't understand use of the word 'regression' to mean 'learn a formula from independant variables to an outcome variable' $\endgroup$ – Aviad Rozenhek Oct 27 '19 at 8:49
  • 1
    $\begingroup$ It more generally means that, but machine learning uses regression but regression is not a machine learning technique, despite popular, incorrect opinion. Statistical learning is separate from machine learning but in general, ML proponents take statistical methods and incorrectly label them as ML so the apparent incongruities pop up. Galton's regression is regression; it has to do with modeling/predicting a tendency. $\endgroup$ – LSC Oct 27 '19 at 9:45
4
$\begingroup$

As opposed to progressing, we are falling back to the mean, i.e. regressing. Hence the term regression ! I think its something that got picked up and stuck.

$\endgroup$
0
1
$\begingroup$

I arrived here via a search for how a regression got its name. Here are the interesting parts of what I found (mostly from wikipedia.)

The term "regression" was coined by Francis Galton in the nineteenth century to describe a biological phenomenon. The phenomenon was that the heights of descendants of tall ancestors tend to regress down towards a normal average (a phenomenon also known as regression toward the mean).7,8 For Galton, regression had only this biological meaning,9,10 but his work was later extended by Udny Yule and Karl Pearson to a more general statistical context.11,12 In the work of Yule and Pearson, the joint distribution of the response and explanatory variables is assumed to be Gaussian. This assumption was weakened by R.A. Fisher in his works of 1922 and 1925.13,14,15 Fisher assumed that the conditional distribution of the response variable is Gaussian, but the joint distribution need not be. In this respect, Fisher's assumption is closer to Gauss's formulation of 1821.

Also very interesting:

In the 1950s and 1960s, economists used electromechanical desk "calculators" to calculate regressions. Before 1970, it sometimes took up to 24 hours to receive the result from one regression.16

Sources

  1. Mogull, Robert G. (2004). Second-Semester Applied Statistics. Kendall/Hunt Publishing Company. p. 59. ISBN 978-0-7575-1181-3.

  2. Galton, Francis (1989). "Kinship and Correlation (reprinted 1989)". Statistical Science. 4 (2): 80–86. doi:10.1214/ss/1177012581. JSTOR 2245330.

  3. Francis Galton. "Typical laws of heredity", Nature 15 (1877), 492–495, 512–514, 532–533. (Galton uses the term "reversion" in this paper, which discusses the size of peas.)

  4. Francis Galton. Presidential address, Section H, Anthropology. (1885) (Galton uses the term "regression" in this paper, which discusses the height of humans.)

  5. Yule, G. Udny (1897). "On the Theory of Correlation". Journal of the Royal Statistical Society. 60 (4): 812–54. doi:10.2307/2979746. JSTOR 2979746.

  6. Pearson, Karl; Yule, G.U.; Blanchard, Norman; Lee,Alice (1903). "The Law of Ancestral Heredity". Biometrika. 2 (2): 211–236. doi:10.1093/biomet/2.2.211. JSTOR 2331683.

  7. Fisher, R.A. (1922). "The goodness of fit of regression formulae, and the distribution of regression coefficients". Journal of the Royal Statistical Society. 85 (4): 597–612. doi:10.2307/2341124. JSTOR 2341124. PMC 1084801.

  8. Ronald A. Fisher (1954). Statistical Methods for Research Workers (Twelfth ed.). Edinburgh: Oliver and Boyd. ISBN 978-0-05-002170-5.

  9. Aldrich, John (2005). "Fisher and Regression". Statistical Science. 20 (4): 401–417. doi:10.1214/088342305000000331. JSTOR 20061201.

  10. Rodney Ramcharan. Regressions: Why Are Economists Obessessed with Them? March 2006. Accessed 2011-12-03.

$\endgroup$
-1
$\begingroup$

"Regression" comes from "regress" which in turn comes from latin "regressus" - to go back (to something).

In that sense, regression is the technique that allows "to go back" from messy, hard to interpret data, to a clearer and more meaningful model. As a physicist, I like the idea, as physicists see natural phenomena as the multiple possible outcomes of a relatively simple natural law.

In other words, the word regression seems to suggest that data is just the visible, tangible effect of a "statistical model". In other words, the model comes first, and your desire is use the data "to go back" to what originated them.

$\endgroup$
3
  • 1
    $\begingroup$ This isn't how the term arose, as other threads explain. $\endgroup$ – Nick Cox Aug 2 '20 at 11:16
  • $\begingroup$ Full disclosure: I confess I did not know the origins of the word in a statistical context when I wrote this. However (the latin meaning is unambiguous and) the somehow flowery "definition" above of why it is used in the context of what we mean by regression, seems equivalent to the one intended by Dalton? $\endgroup$ – famargar Aug 3 '20 at 12:11
  • $\begingroup$ Sorry, but I can't endorse your personal gloss on the word regression as fit for wider use. The origin of the word in Galton's (not Dalton's) work is clear and well documented. It is unfortunate that its original biological meaning has nothing to do with most applications since, but the solution -- if that is seen as a problem -- is to avoid the term, not to attempt unilateral definition. $\endgroup$ – Nick Cox Aug 3 '20 at 13:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.