What is the most suitable optimization algorithm for optimizing maximum likelihood estimator? In excel I used GRG non linear optimization algorithm, is that good enough?
I want to write my own code to understand optimization better so I want to choose most suitable non linear constrained optimization algo for MLE.
Cheers
GRG is a good and robust constrained optimization algorithm. However GRG gives only local solution it may be worthwhile to use an evolutionary solver and obtain the initial estimates and then use the solution obtained using evolutional algorithm as a starting point for GRG to obtain robust optimal solution. You can do this using excel solver.
R has it own library for mle estimation. It also has a function called optim for unconstrained optimization and does not have GRG. You can further check out the optimization cran task view [website][1] for R which has both constrained and unconstrained optimization routines in R
If you post the problem, may be I could provide some additional insights.
I had good experience with stochastic search methods. They basically create a set of markov-chains and sample the parameter space around previous estimates. They work well in situations with objective functions who have multiple local maxima/minima, strong discontinuities or costly function evaluations. Some of the algorithms even work well for stochastic functions.
Easy to implement algorithms include ASA (adaptive simulated annealing), DDS (dynamical dimensioned search), DE (differential evolution) or the adaptive variant JADE (which I use).
More complex algorithms include CMA(-ES) (Covariance matrix adaption) or surrogate assisted techniques like kriging.
