I am trying to minimize an objective function, $J(\theta)$, with respect to $\theta$, a 19-dimensional parameter vector. $J(\theta)$ is a smooth nonlinear function so I have tried various gradient-based optimizers in R to find the solution. A consistent problem is that they all stop optimization early at a point which has a nonzero gradient. The reason for the stop is a "zero step from line search." For example, the code

ucminf(theta = paramGuess, fn = J_obj, control = list(maxeval = 1000, xtol = 1e-15, grtol = 1e-8, trace = 1))

will terminate with the following message:

Line search: alpha = 0.0000e+00, dphi(0) =-8.7877e-02, dphi(1) =-4.9128e-02
Optimization has converged. Stopped by zero step from line search 
maxgradient    laststep     stepmax       neval 
488.2985428   0.0000000   0.4051687  22.0000000  

As you'll notice, the gradient is far from zero. Optim has the same problem.

Is there a way I can control the minimum step size (alpha), so that the optimizer doesn't stop until the maximum element in the gradient is less than grtol?

I have also tried supplying an analytic gradient and this sometimes converges to a solution with a nonzero gradient depending on the initial parameter values. However, sometimes it doesn't, and the optimizer again stops because of "zero step from line search."

Any advice as to why the optimizer is setting the step size equal to zero and how I can prevent this from happening would be greatly appreciated!

  • $\begingroup$ You need to post this on stack overflow, as it's a coding question. I imagine the moderators here are going to close this. $\endgroup$ – user255758 Aug 21 '20 at 3:21
  • $\begingroup$ 19 parameters? I hope you have a massive amount of data that nicely covers the parameter space and that parameters are only weakly correlated. $\endgroup$ – Roland Aug 21 '20 at 6:11
  • $\begingroup$ @Jason_93 I looked carefully at the rules for stack overflow and cross validated and thought of this as more of an optimization question than a coding question. But I will repost on stack overflow if others agree that it should be posted there. $\endgroup$ – EB727 Aug 21 '20 at 13:21
  • $\begingroup$ @Roland To provide more context, $J(\theta)$ is the GMM objective function defined by an economic model that I have written down. The model has 4 equations, each of which was a several parameters to estimate and there are also a few variance parameters to estimate as well. The reason that it is nonlinear is that the same parameters appear in multiple equations, and so GMM is used (instead of simple linear regression on each equation) to take into account the cross-equation restrictions on the parameters. $\endgroup$ – EB727 Aug 21 '20 at 13:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.