# terminology of ordinary least squares

The definition from Wikipedia...

In statistics, ordinary least squares (OLS) is a type of linear least squares method for estimating the unknown parameters in a linear regression model. OLS chooses the parameters of a linear function of a set of explanatory variables by the principle of least squares: minimizing the sum of the squares of the differences between the observed dependent variable (values of the variable being predicted) in the given dataset and those predicted by the linear function.

Is OLS "ordinary" if and only if it is linear? If the answer is "no" then please explain what makes it ordinary.

## 2 Answers

OLS is a type of linear LS (least squares) methods. In OLS, the error variance is constant across samples (homoscedasticity) and also terms are uncorrelated. Other common linear LS methods are WLS (Weighted LS), and GLS (Generalized LS). In WLS, heteroscedasticity is handled, i.e. error variance changes across terms. In GLS, both variable variance and error correlation is handled. In a nutshell, in OLS, error covariance is $$\sigma^2\mathbf{I}$$; in WLS, it is $$\text{diag}(\sigma_1^2,...,\sigma_n^2)$$ (i.e. diagonal but different variances); in GLS it is a full covariance matrix, $$\mathbf{\Sigma}$$, w/o any (that doesn't violate covariance matrix assumptions of course) constraints.

"Ordinary" in the OLS stands for the LS method you use when the residuals in your data are uncorrelated and homoscedastic.

Let $$y=X\beta + \epsilon$$

If $$\epsilon$$, the error term, is uncorrelated and homoscedastic (its variance does not depend on X, i.e. $$E[\epsilon_i^2|x_i]=c$$), then the OLS is unbiased, consistent and efficient.

However, if $$\epsilon$$ is either correlated, or heteroscedastic, or both, the OLS is no longer efficient. In such a case you would use either WLS (weighted least squares) when the error is heteroscedastic, or GLS (generalized least squares) when the error is correlated.

You can read more on this topic either on this wiki page or J.M. Wooldridge "Introductory Econometrics".

• If $E[\epsilon_i^2|x_i]=0$, $\epsilon_i^2$ is a constant (and zero) given $x_i$. – gunes Apr 2 at 9:42
• @gunes You are right, I edited my answer. Thank you for noticing. – Uliana Zaspa Apr 2 at 12:22