Im trying to fit a power addictive nonlinear model, and keep getting a message in the results that convergence criterion met but hessian is singular. What does this mean? also in the correlation matrix I have no values for 1 or more coefficients. My guess is that it means I converged to a local minimum, but I'm not sure.
$\begingroup$
$\endgroup$
1
-
3$\begingroup$ Article: "What to do when your Hessian is not invertible" ;) gking.harvard.edu/files/help.pdf $\endgroup$– ocramzCommented Nov 12, 2015 at 21:52
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
5
The Hessian is the matrix of second derivatives of the objective function you are attempting to minimize (resp. maximize, depending on how SAS set this up). The Hessian is a square $k \times k$ matrix, where $k$ is the number of parameters in your model. In your case, the Hessian is singular, which means that your parameters are linear functions of each other (or almost collinear) ... either absolutely, or with respect to the data that you have.
A young colleague of mine once got the exact error you did in SAS. Turns out he had included an intercept term twice. So the first thing you want to do is to check the specification of the model in SAS.
Another source of difficulty will occur if one of the parameters is basically useless at the data points you have. With a large sample and a good model, the log likelihood should look like a paraboloid near the minimum. However, a useless parameter can produce a long, narrow valley. Any solution in the trough of the valley will produce much the same minimum in the objective function. Your algorithm will terminate normally (it will no longer be able to improve itself by taking additional steps), but the Hessian will be singular.
If you don't have loads of parameters, (and you shouldn't with a non-linear model) you could try plotting the likelihood in the vicinity of the "solution" that SAS came up with, to see where the problem lies.
The Hessian is used to estimate the standard errors of the maximum likelihood estimates, so if the Hessian is broken, you won't get complete entries in the correlation matrix. These are obtained from the inverse of the Hessian, which you can't have, since the Hessian is singular.
I'm not sure that you converged to a minimum. With a singular Hessian, you could have converged to a saddlepoint in the parameter space, or the optimizer may simply have become disorientated and quit in the naïve belief that it had accomplished its mission. Something about your model is not working: either the way you specified it to SAS, or the data you have, or the model you are attempting to fit. Trust nothing in the results you got and start over. Perhaps try a simpler, linear model and then add more complicated features to improve the fit.
-
$\begingroup$ Thanks for the detailed reply! The last part of your last paragraph answers my follow up question, which was "if one of the paramter estimate has all missing values, should I focus on that parameter or revise the entire model?" $\endgroup$– KsimsCommented Nov 12, 2015 at 23:08
-
$\begingroup$ If it's just one variable with missing values, I would remove it and fit the remainder. That one change might fix it. My suggestion in the last paragraph assumed that you did not know where the trouble was, so start small and move up. $\endgroup$– PlacidiaCommented Nov 13, 2015 at 5:19
-
$\begingroup$ Fixed it!! One last question, is there an easy way to tell if I have converged to a local min? After finding my error, I tried a few different transformations and one of them converged after 1 iteration, which makes me think something is not right even though I have no error messages and "singular hessian" is now eliminated! $\endgroup$– KsimsCommented Nov 13, 2015 at 5:47
-
$\begingroup$ The eigenvalues of the Hessian must all be negative. This, in turn, implies that the variances of the parameter estimates will be positive. SAS should issue a warning if this is not the case. At least, this is true when the solution is at an interior point of the parameter space. If you have constraints on a parameter (like it has to be between 0 and 1, or greater than 0), then you need to test the Karush Kuhn Tucker conditions. SAS might offer those as an option. $\endgroup$– PlacidiaCommented Nov 13, 2015 at 14:30
-
$\begingroup$ Regarding constraints, these are only an issue if the solution hits the boundary of a constraint. $\endgroup$– PlacidiaCommented Nov 13, 2015 at 14:31