Best loss function for nonlinear regression I have some nonlinear nonnormal data that I am trying to analyze. The data has been normalized to -1 to 1 and detrended with polynomials of an order of 3. I'm trying to determine if there is a special loss function for nonlinear regression type problems. I did some googling but not much showed up. Right now I'm sticking with MSE.
 A: There's no such a thing as a loss function "for" a particular kind of model. You could be using nonlinear regression with different loss functions. There are many loss functions and you can even construct one yourself. The choice depends on the nature of your problem and the data you are dealing with. Recall that minimizing some loss is equivalent to maximizing a likelihood function (e.g. using squared error is an equivalent of assuming Gaussian likelihood function), so it is tightly connected to the assumptions you are making about the distribution of errors.
More formally, if you think of the model as of something like
$$
y = f(X) + \varepsilon
$$
then the choice of model (e.g. linear regression, nonlinear regression, deep neural network, etc) is related to estimating the expectation $E[y] = f(X)$, while the choice of the loss function impacts how do you treat the residuals $y - f(X) = \varepsilon$.
For example, choosing squared error over absolute error penalizes outliers more, so it would be preferable if this is what you want to achieve. On another hand, absolute error is less prone to outliers, this can be an advantage in another scenario.
The most common choice is defaulting to squared error, though it is somehow an arbitrary choice and doesn't have to be the best in all cases.
A: Other answers (like bdeonovic's and Tim's) discuss "robustness to outliers". I have to admit that while this point of view is extremely common, I do not like it very much.
I find it more helpful to think in terms of which conditional fit (or prediction) we want.

*

*Use the squared errors if you want conditional expectations as fits or predictions. ("Outliers" are then simply observations that are "far away" from the expectation, and which therefore pull the expectation towards them. If your aim is an expectation fit/prediction, then you should think long and hard about whether you want "robustness to outliers", because "outliers" are a fact of life.)

*Use the absolute errors if you want condititional medians as fits or predictions.

*Use quantile (AKA pinball) losses if you want conditional quantiles as fits or predictions.

I have written a short paper (Kolassa, 2020, IJF) on this, in the context of forecasting - but the idea holds in the precise same way for fits.
Thus, I would recommend you think about what kind of fit/prediction you want, and then tailor your loss function to this.
A: Most of the alternative loss functions are for making the regression more robust to outliers. I've seen all of the following in various software package implementations, but I haven't looked too hard into the literature comparing them

*

*least absolute deviation

*least median of squares

*least trimmed squares

*metric trimming

*metric winsorizing

*Huber Loss

*Tukey's biweight loss

*soft L1 loss

*Cauchy loss

*arctan loss

How are you doing the optimization? Did you code it yourself? Are you using Gaus-Newton or Gradient Descent? May want to consider Levenberg–Marquardt (interpolates between Gaus-Newton and Gradient Descent)
