0
$\begingroup$

What is the difference between ordinary LS and linear least squares? From wikipedia seems like linear least squares is a general category that involves using least squares to approximate linear functions, where as OLS is a specific way to do so?

A few places I've seen linear least squares defined:

"linear least squares regression can be used to fit the data with any function of the form $f(\vec{x} ; \vec{\beta})=\beta_{0}+\beta_{1} x_{1}+\beta_{2} x_{2}+\ldots$

in which

1.each explanatory variable in the function is multiplied by an unknown parameter,

2.there is at most one unknown parameter with no corresponding explanatory variable, and

3.all of the individual terms are summed to produce the final function value.

In statistical terms, any function that meets these criteria would be called a "linear function". The term "linear" is used, even though the function may not be a straight line, because if the unknown parameters are considered to be variables and the explanatory variables are considered to be known coefficients corresponding to those "variables", then the problem becomes a system (usually overdetermined) of linear equations that can be solved for the values of the unknown parameters. To differentiate the various meanings of the word "linear", the linear models being discussed here are often said to be "linear in the parameters" or "statistically linear". Why "Least Squares"? Linear least squares regression also gets its name from the way the estimates of the unknown parameters are computed."

https://www.itl.nist.gov/div898/handbook/pmd/section1/pmd141.htm

And then wikipedia states : "Linear least squares (LLS) is the least squares approximation of linear functions to data. It is a set of formulations for solving statistical problems involved in linear regression, including variants for ordinary (unweighted), weighted, and generalized (correlated) residuals. Numerical methods for linear least squares include inverting the matrix of the normal equations and orthogonal decomposition methods. "

"- Ordinary least squares (OLS) is the most common estimator. OLS estimates are commonly used to analyze both experimental and observational data. The OLS method minimizes the sum of squared residuals, and leads to a closed-form expression for the estimated value of the unknown parameter vector $\beta$ : $$ \hat{\boldsymbol{\beta}}=\left(\mathbf{X}^{\top} \mathbf{X}\right)^{-1} \mathbf{X}^{\top} \mathbf{y} $$ where $\mathbf{y}$ is a vector whose ith element is the ith observation of the dependent variable, and $\mathbf{X}$ is a matrix whose ij element is the ith observation of the jth independent variable. The estimator is unbiased and consistent if the errors have finite variance and are uncorrelated with the regressors" https://en.wikipedia.org/wiki/Linear_least_squares

$\endgroup$
4
  • 1
    $\begingroup$ Could you please explain in more detail what "linear least squares" might be?? $\endgroup$
    – whuber
    May 5, 2022 at 2:30
  • $\begingroup$ Ok I added a couple of definitions I've seen $\endgroup$
    – a12345
    May 5, 2022 at 2:43
  • $\begingroup$ The two descriptions look remarkably similar, don't they? $\endgroup$
    – whuber
    May 5, 2022 at 2:48
  • $\begingroup$ yeah they are similar $\endgroup$
    – a12345
    May 6, 2022 at 22:42

1 Answer 1

1
$\begingroup$

No difference at all. These are equivalent.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.