Difference between Bayesian Optimization and Bayesian Statistical Inference [closed]

Question

Can someone give a lucid explanation pointing out how are Bayesian Optimization and Bayesian Statistical Inference different to each other? Can they be seen as complementary to each other or are they overlapping with each other? Any interesting insight regarding their intersection is welcome.

I understand Bayesian Optimization generally refers to Gaussian Process and fitting GPs on your data. Bayesian Statistical Inference involves computing posterior from your data and then use that posterior to compute various expected quantities, or one can just refer to Gelman's book. I have to talk about these terms to a researcher and I don't want to come off as amateurish.

Answer 1

I'll take a go at this.

Bayesian optimization is concerned with finding optimal parameters for a model, with respect to an objective function, by intelligently sampling from the parameter space. A surrogate that approximates the objective function is used by an acquisition function which computes how desirable evaluating the surrogate at a proposed parameter value is. We use a surrogate because the true objective function may be too expensive to compute. The acquisition function is optimized to select the next best parameter value given previous samples 1, 2. This can be contrasted with a more brute-force approach of trying a fixed set of parameters or parameters drawn from probability distributions to minimize or maximize the objective function directly.

Bayesian statistical inference, like frequentist statistical inference, is concerned with estimating unknown parameters of a population using prior knowledge. In the frequentist paradigm, probability is a limiting case of repeated measurements. In the Bayesian paradigm, probability is extended to cover degrees of certainty about statements regarding the unknown population parameters 3.

Both Bayesian optimization and statistical inference use prior information to arrive at an estimate through repeated updates to a joint probability (posterior) distribution given more observations. In Bayesian optimization, the estimate is for optimal parameter values. In Bayesian inference, the estimate is for unknown population parameters.