Reconciling OLS as a linear regression model and polynomial regressors A bit of a naive question.
I understand that OLS is used for a linear regression model (for example, Wikipedia page for OLS:  OLS is a type of linear least squares method for choosing the unknown parameters in a linear regression model (with fixed level-one effects of a linear function of a set of explanatory variables) by the principle of least squares).
I, therefore, don't understand why OLS can be used to estimate what I understand to be non-linear regressions, s.t. $$y= a+bx+bx^2.$$
What am I missing?
 A: You are missing the definition of linear regression. By linear regression, it is meant any regression equation of the form
$$
y = \beta_0+\beta_1x_1+\beta_2x_2+\cdots+\beta_px_p + \epsilon,\tag{*}
$$
where $\beta_i$ are the regression parameters, $x_j$ are the covariates and $\epsilon$ is an error term. The keyword linear has to do with how each $\beta_i$ must enter in (*), i.e. they must enter linearly.
For example
$$
y = \beta_0+\beta_1x_1+\beta_2x_2+\beta_3x_1x_2+\epsilon,
$$
is a linear regression model, whereas the equation
$$
y = \beta_0+x_1^{\beta_1}+\beta_2x_2+\cdots+\beta_px_p + \epsilon, \tag{**}
$$
does not identify a linear regression model. We can still use OLS for (**), though the solution won't necessarily have a closed-form expression as in (*).
A: The method of least squares is a nummerical method for minimizing a function. The solution to any functional form (linear or non-linear) would be $\hat{x} = (A^{T}A)^{-1}A^{T}\beta$. The term OLS is often times used to describe both the nummerical method and the concept of linear regression, however they are different. You could technically solve the linear regression with any type of minimization algorithm (like gradient boosting, for example) to find the minimal residual error. In OLS, however, the least squares method give you the best linear unbiased estimate.
