Is it posible to obtain the next point without bruteforcing in Bayesian Optimization?

The implementations I have found till now of Bayesian optimization tipically find the optimal input point X for next step, which minimizes the output Y of the acquisition function which is being modeled, by testing a lot of samples (by random search, gridsearch, etc) and then selecting the best one. For example, this happens in this implementation, exactly in the acq_max() function located in this file

Aren't there currently implementations where these new points are obtained directly by finding the local minima of the modeled function, maybe with the derivatives of dXi/dY of the corresponding acquisition function (either it is a Gaussian Process, a multi-layer perceptron...)?

Is this even posible? I found some tracks that suggest me that this is posible, like the inversion of MLP, but I can not find this implemented in a bayesian optimizer

• If I understand your question correctly, the package you link to does this as you can see github.com/fmfn/BayesianOptimization/blob/master/examples/… here. However it needs some data to get started, hence the two random points. Could you clarify your question? Jan 8 '18 at 23:03
• This package explores the acquisition function by using the function acq_max located in this file github.com/fmfn/BayesianOptimization/blob/master/bayes_opt/… Jan 9 '18 at 8:48
• Bayesian optimization via Gaussian processes is one way to model response surfaces and therefore find their optima, but LIPO and related methods show that it's not necessary to build a high-fidelity simulation of the entire surface -- you just need to bound the maxima or minima. Then, the procedure reduces to identifying regions with optima that offer improvement. See: stats.stackexchange.com/questions/193306/…
– Sycorax
Jul 11 '20 at 3:15

• Most of the Bayesian optimisation packages I've seen use Gaussian processes for constructing the acquisition function $f$. I think that makes it hard to write down an explicit formula for $f$ because the real $f$ is latent and has a parameter of infinite dimension. With a particular choice of a simple covariance function (e.g. linear or polynomial) this is probably possible. I'm not sure about whether this is implemented in the Bayesian optimisation packages but I'd guess it's probably equivalent to some sort of method in the design of sequential experiments. Maybe worth a new question? Jan 10 '18 at 8:59