2
$\begingroup$

I'm trying to understand the process of Bayesian optimization of a black box function and the bit I'm confused about is how to make the very last prediction of the true maximum after you have made all the function evaluations you can afford to make.

I reviewed the code for two Python implementations:

and in both, the final estimate is simply whichever parameter values resulted in the highest previous actual function value.

My question is, would it not be better to find the maximum of the final gaussian process model (the maximum mean prediction) and use that as the best estimate of the peak of the true function?

To illustrate the question, here is the result of Bayesian optimization of a simple 1-D function after 7 steps using the bayes_opt package noted above.

Matplotlib figure with two subplots showing the Gaussian Process estimates and the value of the utility function over the range of x values

Full code is here if you need to see it.

The best estimate based on the highest actual function evaluation so far is:

sample_max_x = optimizer.max['params']['x']
sample_max_x, function_to_maximize(sample_max_x)
# (2.203, 5.574)

The best estimate based on the peak of the current GP model is:

mu, sigma = posterior(optimizer, x_obs, y_obs, grid=X.reshape(-1, 1))    
posterior_max_x = X[mu.argmax()]
posterior_max_x, function_to_maximize(posterior_max_x)
# (2.475, 5.713)

The true maximum is (2.509, 5.714), so obviously, in this case taking the maximum of the GP model is the best estimate but is it always a good idea?

Or is there too much risk given the uncertainty that the maximum of the GP model might be far worse than the best sample so far?

$\endgroup$
0

2 Answers 2

1
$\begingroup$

The value of the target function at the maximum of the GP model will not always be larger than the largest observed value.

The posterior at posterior_max_x has quite a bit of uncertainty. Before it was evaluated, there was some chance of the value of the target being larger than function_to_maximize(sample_max_x) and some chance of it being smaller. If you believe the confidence intervals in the plot, you can use them to quantify this risk. But in realistic problems, the confidence intervals in the GP may not reflect the full uncertainties.

There's no technical reason that the library couldn't report posterior_max_x instead of sample_max_x, but it raises the question of how much risk the user wants to take of the answer being wrong. If you only report the best value of those that have actually been evaluated, the user will know function_to_maximize(sample_max_x) precisely rather than having an uncertain answer. That makes using the library much simpler and less confusing the the user.

$\endgroup$
2
$\begingroup$

I realize I read it too quickly and mis-interpreted the question. I think the top answer already addressed the more specific question adequately, but yeah - typically the best observed point is used because there is no surety that the predicted GP maximum is higher than the best current value. It may be, but you know the high performance at the best point you've observed.

Going with somewhere you haven't observed yet runs the risk of the GP being inaccurately high and ended up with poorer performance. If you have the computational time, you could always do one last greedy step - though a new GP conditioned on that as well may have a new predicted maximum. In general I think the reason to not use the predicted max is out of precaution against the GP just being off.

Old answer with mis-interpreted question:

For Bayesian optimization, using the prediction function in a "greedy" sense - next evaluating at the maximum predicted value - can lead to poor optimization performance through inadequate exploration.

Part of the value of the Bayesian optimization approach is that it balances attempting to exploit regions of high predicted performance and explore regions of high uncertainty. A model can become stuck in a local optimum if it predicts high performance there and then only searches near there.

"A Taxonomy of Global Optimization Methods Based on Response Surfaces", Jones (2001) does an exploration of approaches including comparison of greedy selection with expected improvement and some other approaches and finds the greedy methods can suffer from the issues described above.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.