Does the use of OLS imply the model is linear in parameters?

Question

I used to say that OLS is an estimation technique and should never be confused with the type of model on which it is applied. Thus a phrase like "I have an OLS model" would not make sense to me, strictly speaking. (I would usually be able to guess what people mean, though.) However, in a comment under this post Dave offers a point to the contrary (if I am interpreting it correctly):

I’m not actually sold on OLS referring to an estimation technique rather than a model. Yes, we can estimate the coefficients of a linear model many ways and can apply minimization of square loss to estimating the coefficients of a nonlinear regression. However, the “ordinary” in OLS suggests to me an interest in a linear model whose coefficients are estimated by minimizing square loss.

Wikipedia's article on OLS seems to contain a similar message:

In statistics, ordinary least squares (OLS) is a type of linear least squares method for choosing the unknown parameters in a linear regression model (with fixed level-one effects of a linear function of a set of explanatory variables) by the principle of least squares: minimizing the sum of the squares of the differences between the observed dependent variable (values of the variable being observed) in the input dataset and the output of the (linear) function of the independent variable.

(emphasis is mine)

So strictly speaking, does the use of OLS imply the model is linear in parameters? In other words, does the term OLS refer to an estimation technique and linearity of the model at once?

I would like the answers to focus on the theoretical aspects of the issue. The fact that a lot of people misuse statistical terms such as OLS is of less concern for me.

Answer 1

Ordinary least squares regression is a special case of least squares regression.

With least squares regression we try to find a fit $\hat{y}_i({\bf{x}}_i,\boldsymbol{\beta})$ to datapoints $y_i$ by minimising the sum of (weighted) squared residuals.

$$\text{given data $\bf{x}_i$ and $y_i$, and weights $w_i$ find $\boldsymbol{\beta}$ that minimises:} \quad L = \sum_{i = 1}^n w_i [y_i-\hat{y}_i({\bf{x}}_i,\boldsymbol{\beta})]^2$$

OLS is the special case when the weights are equal $w_i = 1$ and the model is a linear combination

$$\hat{y}_i({\bf{x}}_i,\boldsymbol{\beta}) = \beta_1 f_1({\bf x}) + \beta_2 f_2({\bf x}) + \dots +\beta_p f_p({\bf x}). $$

OLS is by definition using a linear model.

---

But not all methods that use linear models are OLS. For instance think of GLM, quantile regression, lasso/ridge or Bayesian modelling, which can use a linear model but with a different cost function.

Answer 2

By means of OLS one can estimate non linear relations provided they are purely additive or purely multiplicative (log-additive). For instance, quadratic relations like this: $$ y_t = a + bx_t + cx^2_t + e_t, $$ are perfectly fitted within the OLS framework and of course, their variants and extenions; but not like this: $$ y_t = a\log b^{2x_{1,t}}+ cx_{2,t}^{b}e_t. $$

The latter can arise in structural specifications where typically the parameters a,b,c is what you can get from data, so that reduced version models are simply ill designed to make good forecasts.