Why would I need this?

I got an idea about a new optimization scheme for bootstrapping. To motivate it's usage I need simple motivating examples to compare to other methods and show that it works. These examples should have some of the properties below.


I am looking for an example of an ill-conditioned likelihood function for doing maximum likelihood estimation.

This likelihood function should have the following properties:

  • Simple in a sense that it does not stem from a complicated model, it can be described very briefly

  • Traditional optimization methods fail or are hard to apply, especially because the Hessian is ill-conditioned, i.e. Newton's method would not work except close to the optimum. Close to the optimum, meaning that if I pick a random starting value for the optimization, the Newton method will fail most of the time.

  • Most current solutions for this problem rely on a heuristic or non-classical optimization techniques/methods to get close to the optimum.

  • Bootstrapping makes the optimum move so far away that starting at the MLE estimates, traditional optimization methods (such as Newton's method) fail. This is the most important requirement.

  • This can also be a function corresponding to an unconstrained optimization problem that does not necessarily need to be a likelihood, just something where bootstrapping of parameters is performed. Something like an ecological or biological model of some sort.

  • The function should preferably be unimodal. This is also an important requirement.

  • The example problem can also be a reparameterization of a problem that is easy to solve, but becomes hard when the alternative parameterization is used.

Are there simple examples that satisfy these criteria out there? I have been looking for one for some while now and I do not manage to find a simple example. Hope that someone here can provide one!

  • $\begingroup$ Must it be an unconstrained optimization problem? A problem that I have worked on a lot meets all of these criteria except that you are trying to optimize over a probability vector $p$ (whose length would actually change for each bootstrap sample). Technically, you could reparameterize it to be an unconstrained problem, but your algorithm would be guaranteed to fail as the solution typically is on the boundary (i.e. $\hat p_j = 0$ for many values of $j$). $\endgroup$ – Cliff AB Oct 22 '15 at 17:44
  • $\begingroup$ @CliffAB you can add it as an answer, most of these criteria should preferably be met, although they are not forced. I am trying out a new method I am developing and I need some examples to run it on. They should preferably be simple to describe. Maybe I can tweak it to work on your problem. $\endgroup$ – Gumeo Oct 22 '15 at 17:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.