# When is Maximum Likelihood the same as Least Squares

In this paper on p315:

http://www.ssc.upenn.edu/~fdiebold/papers/paper55/DRAfinal.pdf

They explain that they use Levenberg Marquardt (LM) (along with BHHH) to maximize the likelihood. However as I understand it LM can only be used to solve Least Squares (LS) problems? Are the LS and MLE solutions the same for this type of problem?

I know that when the errors are normal like in OLS then the solutions are the same. Here the processes being estimated are AR(1) so the errors are normal even though the overall process is not. Can I still treat the MLE and LS solution interchangeably in this situation?

In which case can I just apply LM to solve the the LS solution safe in the knowledge that the optimal LS parameters are also the ones that will solve the MLE problem?

Or does the LM algorithm have to be changed in some way so that it can be applied directly to the MLE estimation? If so how?

Kind Regards

Baz

## 1 Answer

Levenberg-Marquardt is a general (nonlinear) optimization technique. It is not specific to LS, although that is probably its widest use. Looking at your referenced paper, they are (mostly) fitting state-space models with additive Normal errors. Forming the likelihood function yields

$\ln L(\theta|X=x) = K - \ln \sigma -\frac{1}{2\sigma^2}\sum\left(x_j-f(\theta)\right)^2$,

which is a nonlinear least squares problem.