# Using L-BFGS-B optimization

This is a general question about how the L-BFGS-B optimization algorithm works.

I have encountered some strange likelihoods in a model I was running (which uses optim from R, and the L-BFGS-B algorithm). The function optimizes over a parameter, which is constrained to 0-1 and maximizes the likelihood (Minimizes the negative log-likelihood I believe is what it technically does)

As I said earlier, I found some odd results and since the model does not take long to run, I ran all possible values of the parameter and plotted the Likelihood for each value, the results looks like so: The red line indicates the value that optim suggested, and the blue line is the max value I found by doing all possible values.

My question is then why is this happening?

Does the L-BFGS-B employ some kind of Newton-Raphson search algorithm, ie. it only finds local maxima / minima?

Or does the systematic process it goes through just not fit in a situation where the likelihood probability is as seen in the graph?

Thanks,

Sam

• ML depends only on one parameter? You say that it is constrained to interval [0,1], yet in the graph we see values from 0 to 50. If you do minimisation, then the minimum should be achieved at point (30,-210), are you sure you did not confuse minimisation with maximisation? From what I see, you either confused minimisation with maximisation, or your optimal solution lives on a boundary. In the latter case gradient based methods are not guaranteed to work. Sep 23, 2013 at 7:16

If this assumption is correct then the problem is that the optimal solution is on the boundary of the parameter space - i.e. from your exhaustive search the best value of the parameter is $1$. I suspect this may be compounded by your choice of starting value (is it $0$?).
I find it helpful in diagnosing issues with optim to include a print statement in the score function so I can follow which parameter values are being used at each function call. This may give you an intuition as to why it is not exploring certain regions of the parameter space.
• The score function is the function to be minimized, in your case the negative log-likelihood function. Judging from the points in your plot it looks like if you start at $0.5$ then the gradient will cause it to head straight towards $0$. I would assume if you changed the starting value to, say, $0.75$ you would end up with the solution at $1$. One final point of clarification, is this really a one-dimensional optimization? In which case the optimize function may be more appropriate - but still wouldn't solve the issue with the solution lying on the boundary. Sep 25, 2013 at 7:41
• One final suggestion - you could try transforming the parameter to avoid using constrained optimization and move the optimal solution away from the boundary. You optimize between $-\infty$ and $\infty$ and apply a transformation such as sin before evaluating the negative log-likelihood. Specifically you'd do (sin(x)+1)/2 to have it lie in [0,1]. Sep 26, 2013 at 7:31