When I fit the mixed effect model I got this warning:

Warning messages:

1: In checkConv(attr(opt, "derivs"), opt\$par, ctrl = control\$checkConv, : Model failed to converge with max|grad| = 0.00202181 (tol = 0.001, component 1)

2: In checkConv(attr(opt, "derivs"), opt\$par, ctrl = control\$checkConv, : Model is nearly unidentifiable: very large eigenvalue - Rescale variables?

I think it is telling me the optimization does not converge, (gradient is not close zero). So I listed more details about the optimization.

enter image description here

Could any one tell me why we have a large number in Hessian matrix? what is happening there?

  • $\begingroup$ Have you rescaled the variables? $\endgroup$ – JimB Jan 3 '18 at 19:19
  • 1
    $\begingroup$ I second @JimB in rescaling the values. lme4 hates it when your variables are on very different scales. Sometimes REML can get hung up in the parameter space and not converge, too. I might try using brms to run a Bayesian model (needs a C++ compiler, but has very similar API to lme4). Give it sensible priors, and they should nudge the algorithm to find a best solution in the parameter space. $\endgroup$ – Mark White Jan 3 '18 at 21:06
  • $\begingroup$ @MarkWhite and JimB thanks for the comments. I scaled, and it DID converge. But I just want to know what happened on non converge case: why large values in Hessian is a bad thing? Numerical issues with IEEE754? $\endgroup$ – Haitao Du Jan 3 '18 at 21:11
  • $\begingroup$ it's a good question; I wonder myself. It was one of those things that people hand-waved over whenever I was taught multilevel modeling. I'm afraid I don't know enough calculus to understand the Hessian matrix terribly well... $\endgroup$ – Mark White Jan 3 '18 at 21:14
  • 1
    $\begingroup$ If you're training some kind of logistic regression, then the large hessian could be the result of a large feature pushing you really close to to 0 or 1, which can blow up your loss function's derivatives. $\endgroup$ – Alex R. Jan 4 '18 at 0:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.