# What is ordinary, in ordinary least squares?

A friend of mine recently asked what is so ordinary, about ordinary least squares. We did not seem to get anywhere in the discussion. We both agreed that OLS is special case of the linear model, it has many uses, is well know, and is a special case of many other models. But is this really all?

Therefore I would like to know:

• Where did the name really come from?
• Who was the first to use the name?
• It is ordinary because there are other variants now. Weighted. Robust. nonlinear. ... Ordinary is how you keep it from being confused with something else. May 8, 2016 at 12:22
• As a guess OLS is likely the first numerical fit algorithm developed circa 1805 AD. The name ordinary was likely added after variations on the theme were developed.
– Carl
Nov 18, 2016 at 23:29

Least squares in $$y$$ is often called ordinary least squares (OLS) because it was the first ever statistical procedure to be developed circa 1800, see history. It is equivalent to minimizing the $$L_2$$ norm, $$||Y-f(X)||_2$$. Subsequently, weighted least squares, minimization of other norms (e.g., $$L_1$$), generalized least squares, M Estimation, bivariate minimization (e.g., Deming regression), non-parametric regression, maximum likelihood regression, regularization (e.g., Tikhonov, ridge) and other inverse problem techniques and multiple other tools were developed. There is still controversy over who first applied it, Gauss or Legendre (see link). The term "ordinary" (implying in $$y$$) was obviously added to "least squares" only after so many alternative methods arose that the (still most) popular OLS needed to be differentiated from the plethora of other minimizations that became available. The word ordinary is often used in mathematical jargon as a synonym of simple. For example, consider the phrase ordinary differential equations. When exactly adding ordinary$$+$$least squares occurred would be hard to track down since that occurred when it became natural or obvious to do so. EDIT (for @alexis): Early 20th century I would presume, @GeoMatt22 says 1921 in comments below, but the mathematical meaning of ordinary (sive, vanilla) predates that and certain differential equations were called ordinary in the late 18th century as per The History of Differential Equations, 1670--1950.
• Its actually in there somewhat already under "other" norms, for example, $||1-\frac{Y}{f(x)}||$ is weighted.
• I have never heard "least squares" used to describe minimization that is not an $L_2$ norm. (Note that "energy norms" such as used in Tikhonov regularization are just an $L_2$ of a transformed vector.) A more general term would be just "regression", or perhaps "M estimation". Dec 17, 2016 at 6:11