# Non-linear least squares and irregular term conditions

I know that when estimating and using ordinary least squares, irregular terms have to be iid N(0, σ2 ). Now I want to use non-linear least squares method for estimations. Are there any specific conditions that irregular terms have to fullfill? Or do they have to be still iid N(0, σ2 )?

• The errors don't have to be normal to fit a model via least squares, but as you move away from normality you'll start to lose efficiency, and the usual inferential calculations (such as hypothesis tests or confidence intervals) rely on it. – Glen_b Dec 16 '16 at 23:14
• @Glen_b there may be a disconnect between math/stats communities here, but I commonly see "pseudo-normal" style error estimates for (local) nonlinear optimizations*, which to me seemed simply based on the idea of a local quadratic model, which should be positive-definite at a local minimum. (*i.e. not even necessarily least squares, though at least 2nd-order smooth one would hope! seems to be commonly built into optimization libraries as an output.) – GeoMatt22 Dec 17 '16 at 6:56
• (continued) The connection being that a multivariate Gaussian negative log-likelihood is simply a positive-definite quadratic. The main issue I have seen with this is commonly optimization Hessian results for massively non-convex problems will be presented as "the uncertainty", when really they are just the "local sensitivity" of the objective function to the parameters. – GeoMatt22 Dec 17 '16 at 7:09
• Does my answer resolve your question? – Mark L. Stone Dec 20 '16 at 20:38

Similarly, for nonlinear least squares, errors having zero mean Multivariate Normal with general covariance matrix can be handled efficiently by using generalized nonlinear least squares. Specifically, let $C$ be that covariance matrix, and let $R$ be the upper triangular Cholesky factor of $C$, i.e., $R' R = C$. Let $Y$ be the column vector of LHS values, and $f(X)$ be the column vector of RHS values (fitted function), so that $Y - f(X)$ is the column vector of residuals. Generalized nonlinear least squares can be conducted as ordinary nonlinear least squares applied to the transformed residuals $R^{-T}(Y - f(X))$. This works because the transformed residuals are iid $N(0,1)$. When this procedure is used, it is equivalent to minimizing $(Y - f(X))^TC^{-1}(Y - f(X))$.