I am looking for a statistical method (and a link to a nice R package would be cool too!) which allows me to find which point to evaluate next for a given function.

I have a non-stochastic function z := f(x, y), which can be evaluated for given x, y pairs. Evaluation is expensive so I want to optimize which points to evaluate next to reduce number of function calls.

My goal is to get a good approximation of the surface of z over the grid of possible x and y values and not like many optimization methods only find the maximum (or minimum) value.

So basically I need to define some objective function that tells me that points far away from previously evaluated points and regions with a high variation in z should be prioritized, while regions in which z is nearly constant and close to previously evaluated points should have a low priority. So a combination of distance and variation and then find the maximum of this function, which is then the next point to evaluate z on.

Gaussian Processes / Bayesian Optimization could be fitting, but I am not sure how exactly I can make it work in my example.

  • $\begingroup$ In Bayesian optimization, there are several methods of choosing the next sample, many of which take the form of a optimization subproblem. However these functions are using trying to balance improving the objective and exploring the function space. So you're just interest in finding the area of maximum variance. If this doesn't exist in a single package, I'm sure you can use GP regression (the basis of Bayesian optimization) and optimization packages in concert to produce the functionality you require. $\endgroup$ – deasmhumnha Aug 2 '18 at 19:26
  • 1
    $\begingroup$ Bayesian optimization isn't really the right framework because its goal is to minimize a function, whereas your goal is to approximate the function. Bayesian optimization doesn't care if its approximation is poor in a region that's unlikely to contain the minimum. $\endgroup$ – user20160 Aug 3 '18 at 6:00
  • $\begingroup$ Thank you for your comments! Yeah, I was wondering if BO could be tweeked to do something else than pure minimization. What could I use instead, any ideas? $\endgroup$ – needRhelp Aug 3 '18 at 17:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.