Assume we have a set of samples and estimate the underlying distribution with a non-parametric density estimator like the Kernel Density Estimator. Lets assume with a gaussian kernel.

In my case it is highly non-convex and multimodal. My goal is to find this distribution's peak. I would handle this by sampling from the distribution itself. Maybe in combination with a gradient ascent to be sure ending up in local maxima.

What are state of the art methods to find the peak of such a density?

  • $\begingroup$ See my answer here. $\endgroup$ – lacerbi May 14 '16 at 21:32
  • $\begingroup$ Thanks. But I think in my case it's slightly different. As far as I know a non-parametric density is not differentiable (that's a similarity). But it's not a black box as I can sample directly from the density (or at least approximate a proper sampling). $\endgroup$ – Chris May 15 '16 at 16:16
  • 1
    $\begingroup$ Sure, you have more information. (Also, if your kernel is differentiable, the KDE will be differentiable too as a finite sum of kernels, I don't see why not?). You can use samples from your distribution to initialize CMA-ES parameters (starting mean and covariance matrix), or the initial points of some other population-based method (e.g., genetic algorithms, particle swarm optimization), but I would try CMA-ES first (since you asked about state-of-the-art). The only catch with CMA-ES is that it needs a lot of function evaluations, but I assume that evaluating the KDE is not very costly. $\endgroup$ – lacerbi May 15 '16 at 16:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.