1
$\begingroup$

So I know there are several types of acquisition function for Bayesian optimization technique. But according to Wikipedia

There are several methods used to define the prior/posterior distribution over the objective function. The most common two methods use Gaussian processes in a method called Kriging. Another less expensive method uses the Parzen-Tree estimator to construct two distributions for 'high' and 'low' points, and then finds the location that maximizes the expected improvement.

So is the expected improvement a Gaussian process, or is it something different?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Expected improvement (EI) is a type of acquisition function. However, it relies on a surrogate model which often is defined as Gaussian processes. So, they are different things. While the surrogate model helps to represent the underlying (true) objective function of the problem, acquisition function like EI is oriented to help in selecting the next point to evaluate in the true objective function.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Is the acquisition function should be maximize or minimized (in this case EI)? Whats the theory behind it? $\endgroup$
    – Lahiru
    Commented Jun 17, 2022 at 5:22
  • 1
    $\begingroup$ Hi @Lahiru! The objective of the acquisition function is to facilitate the selection of the most promising solution in the next iteration. Therefore, what we intuitively seek is to maximize that criterion. In the specific case of Expected Improvement, which is based on the probability of improving the current estimate, it is clear that we would also need to maximize this AF. In the following link you will find more about the theory behind AFs: ekamperi.github.io/machine%20learning/2021/06/11/… $\endgroup$
    – Pavel Novoa
    Commented Jun 18, 2022 at 7:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.