# The best line fit can be found analytically by the least squares method. So can we say that linear regression (least squares) has an optimizer?

The best line fit can be found analytically by the least squares method. So can we say that linear regression (least squares) has an optimizer?

For example, for logistic regression I can use an optimizer, gradient descent. But for linear regression, where the best model parameters can be found analytically, does it make sense to say that I use an optimizer? An optimizer in general can be any algorithm that takes as input a model, model parameters, evaluator (function that tells you how good particular parameters are), and produces as output the best model parameters.

• Well you can always fit linear regression using gradient descent, which can be computationally beneficial, e.g. when design matrix $X$ is very large May 21, 2017 at 16:49
• Your question actually hinges on your definition of "best". Without defining it, you can't even tell if you're optimizing by that definition. Also beware conflating the optimization method with the criterion you're optimizing; the two are distinct. May 21, 2017 at 17:02

When we solve a least square problem, no matter which algorithm to use, we are doing an optimization: $\text{minimize}~ \|Ax-b\|^2$.