GLMM optimiser test - optimx.L-BFGS-B doesn't converge, but the rest do

Question

I am running GLMM using lme4 in R for the first time. I have a complex model (with three main effects and four interactions), as well as a random intercept of participant (as my experiment used a repeated measures design).

Even with this basic random effects structure, I have been having issues with convergence. The model works with the bobyqa optimiser, but I am concerned it may be changing the data and giving me unreliable estimates.

As suggested by the authors of lme4 (on "?converge"), I have checked all possible optimisers. This is what it says re. convergence:

try all available optimizers (e.g. several different implementations of BOBYQA and Nelder-Mead, L-BFGS-B from optim, nlminb, ...) via the allFit() function, see ‘5.’ in the examples. While this will of course be slow for large fits, we consider it the gold standard; if all optimizers converge to values that are practically equivalent, then we would consider the convergence warnings to be false positives.

These are the results I am getting:

bobyqa TRUE
Nelder_Mead TRUE
nlminbw TRUE
optimx.L-BFGS-B FALSE
nloptwrap.NLOPT_LN_NELDERMEAD TRUE
nloptwrap.NLOPT_LN_BOBYQA TRUE

So all but one of the models are fitting. The estimates from these are very similar to each other. See the example below.

Is this okay or is this problematic? If this is okay, should I report the results with the bobyqa optimiser? If this is problematic, what analysis would you recommend? ANOVAs are sub-optimal as the data is binary (correct/incorrect), and logistic regression is sub-optimal as the design is repeated measures.

Thank you for your help. Hopefully I've explained the problem in enough detail, but I'm happy to provide more information if necessary.

Thanks.

Answer 1

Don't worry about the single non coverging optimizer: there is no free lunch and no optimizer that works best in all situations. This is why the lme4 team has built-in support for several different optimizers (and you can even use your own custom optimizers).

All of your estimates are fairly close in numeric terms, but just how close they are will depend on your particular dataset and domain.

In terms of which optimizer to use, I would tend to prefer the fit with the best log-likelihood (i.e. biggest -- for negative values, this means closest to zero) because for identical models but with different approximations (here: optimizers), the best log-likelihood indicates the best approximation.

My guess is that bobyqa won't have the best log-likelihood because it is closer to the non-converged than the converged estimates, at least for the first two predictors.

One final thing: the optimizers don't "change" the data. They simply try to iteratively solve the problem you give them (via your model specification). The different optimizers are just different approaches for how to handle each iteration, and the convergence tests largely work by checking how the approximate solution changes from iteration to iteration. In addition to the differing performance between optimizers, which can lead to some truly performing better or worse in a given situation, these convergence tests are themselves approximate and not perfect, so they can also sometimes give false positives. A lot of work has gone into tuning these tests so that they give false negatives hopefully very rarely and false positives not all too often.