Estimating errors for parameters from a nonlinear fitting procedure

Question

I'm examining a code in C++ for a nonlinear fit. It is basically a Levenberg Marquardt routine you can find on Netlib or elsewhere. The last step is estimating the errors of the parameters that are fitted. From literature, I know that the variance of my parameters, $a_j$, is given by $\sigma^2 a_j = C_{j,j}$ where $C$ is the inverse of the Hessian. In the code this is exactly what happens, i.e. by a Gauss-Jordan algorithm the Hessian is inverted. What puzzles me is the last step to calculate the relative error:

$\delta a_j = \sqrt{\frac{f(x, \vec{a}_\text{fit})}{50 - N} \vert C_{j,j} \vert } \frac{1}{a_j}$

The value $f$ is the minimum value, i.e. the minimum sum of squares. $N$ is the number of parameters that are to be fitted. $\vert C_{j,j} \vert$ is the absolute entry in the inverted Hessian. $a_j$ is the parameter for which the error is to be estimated.

From a dimensional point of view, the result is a percentage, which is fine, but the denominator $50 - N$ puzzles me. At the end, there's a test whether $\delta a_j$ is larger than 50 %, but I can't see that this is connected with the shown calculation. If someone has an idea of the denominator, thanks for a hint.

Answer 1

Assuming that $f(x,a_{fit})$ is the sum of squares of the residuals, then

$s=\sqrt{\frac{f(x,a_{fit})}{50-N}}$

is an estimate of the standard deviation of the measured data points. The denominator of $50-N$ takes into account that $f$ is the sum of the squares of 50 residuals. You're assuming here that each of the 50 measurements has normally distributed errors that are independent of each other wish standard deviation $s$. The $-N$ in the denominator accounts approximately for the fact that you've fit $N$ parameters to the data. If you didn't include the $-N$, then you'd be effectively over fitting the data and underestimating $s$.

This is discussed in many books that cover nonlinear regression. A fairly elementary treatment is in the book by Bevington and Robinson.

You should be aware that there are many approximations involved with this approach to estimating a confidence interval for the fitted parameter. It's easy to find nonlinear regression problems for which those approximations don't work well. You'd be well advised to to test these approximations before you use them. Bevington and Robinson discuss many of the problems that can occur.