Return to Answer

Edits: Reorganized and expanded my answer to make better sense.

The empirical result here is that having access to a posterior, even if partially misspecified, generally produces better results than other methods. (There are caveats and situations in which Bayesian optimization is no better than random search, such as in high dimensions.)

Edits: Added reference to this paper in the body, as requested by the OP.

The empirical result here is that having access to a posterior, even if partially misspecified, generally produces better results than other methods. (There are caveats and situations in which Bayesian optimization is no better than random search, such as in high dimensions.) In this paper, we perform an empirical evaluation of a novel BO method vs. other optimization algorithms, checking whether using BO is convenient in practice, with promising results.

Bayesian optimization uses the posterior over $f$ (in particular, the joint conditional posterior mean and variance at any point) to guide the search of the (global) optimum via some principled heuristic. The classical choice is to maximize the expected improvement over the current best point, but there are even fancier methods, like minimizing the expected entropy over the location of the minimum.

Bayesian optimization uses the posterior over $f$ (in particular, the joint conditional posterior mean and variance at any point) to guide the search of the (global) optimum via some principled heuristic. The classical choice is to maximize the expected improvement over the current best point, but there are even fancier methods, like minimizing the expected entropy over the location of the minimum (see also here).

I am probably being a naive empirical Bayesian here. I am going with the mantra that quantification of uncertainty is generally better than no uncertainty, which may sound like a theoretical dogma but I found out to hold empirically (somewhat to my surprise).

The posterior distribution allows you to do computations that you simply cannot do with a straightforward MLE. The simplest case is that today's posterior is tomorrow's prior. Bayesian inference naturally allows for sequential updates, or more in general online or delayed combination of multiple sources of information (incorporating a prior is just one textbook instance of such combination). Bayesian Decision Theory with a nontrivial loss function is another example. I would not know what to do otherwise.

This is all good and known but the specific example I have in mind to make my case (and that convinced me that being Bayesian is a good thing, not just a faction that I happened to like) is Bayesian optimization, see e.g.,

Shahriari, Bobak, et al. "Taking the human out of the loop: A review of Bayesian optimization." Proceedings of the IEEE 104.1 (2016): 148-175.

or more in general probabilistic numerics,

Hennig, Philipp, Michael A. Osborne, and Mark Girolami. "Probabilistic numerics and uncertainty in computations." Proc. R. Soc. A. Vol. 471. No. 2179. The Royal Society, 2015.

In the case of Bayesian optimization, the key idea is to have a posterior over a function (e.g., via a Gaussian process) as opposed to a point estimate (e.g., as you would get by fitting a radial basis function network) and to use that posterior -- in particular, the conditional posterior mean and variance at each point -- to guide the search of the (global) optimum via some principled heuristic (the classical choice is to maximize the expected improvement over the current best point, but there are even fancier methods, like minimizing the expected entropy over the location of the minimum).

Crucially, having access to a posterior, even if (largely) misspecified, seems to produce better empirical results in a very practical, no-nonsense endeavor such as optimizing a function.  (There are caveats and situations in which Bayesian optimization is no better than random search, such as in high dimensions.)

So, what's going on here? I guess that the underlying assumption here is that the model is wrong, but not that wrong, such that the answers that we get might be suboptimal (i.e., we might be throwing away good information), but still better than with no uncertainty information.

Why does it work at all? I don't have an answer. Since you put the "philosophical" tag... I guess we are entering the realm of the problem of induction, or the unreasonable effectiveness of mathematics in the statistical sciences (specifically, of our mathematical intuition & ability to specify models that work in practice) -- in the sense that from a purely a priori standpoint there is no reason why our guesses should be good or have any guarantee (and for sure you can build mathematical counterexamples in which things go awry), but they turn out to work well in practice.

Edits: Reorganized and expanded my answer to make better sense.

I am giving an answer as a naive empirical Bayesian here.

First, the posterior distribution allows you to do computations that you simply cannot do with a straightforward MLE. The simplest case is that today's posterior is tomorrow's prior. Bayesian inference naturally allows for sequential updates, or more in general online or delayed combination of multiple sources of information (incorporating a prior is just one textbook instance of such combination). Bayesian Decision Theory with a nontrivial loss function is another example. I would not know what to do otherwise.

Second, with this answer I will try and argue that the mantra that quantification of uncertainty is generally better than no uncertainty is effectively an empirical question, since theorems (as you mentioned, and as far as I know) provide no guarantees.

Optimization as a toy model of scientific endeavor

A domain that I feel fully captures the complexity of the problem is a very practical, no-nonsense one, the optimization of a black-box function $f: \mathcal{X} \subset \mathbb{R}^D \rightarrow \mathbb{R}$. We assume that we can sequentially query a point $x \in \mathcal{X}$ and get a possibly noisy observation $y = f(x) + \varepsilon$, with $\varepsilon \sim \mathcal{N}(0,\sigma^2)$. Our goal is to get as close as possible to $x^* = \arg\min_x f(x)$ with the minimum number of function evaluations.

A particularly effective way to proceed, as you may expect, is to build a predictive model of what would happen if I query any $x^\prime \in \mathcal{X}$, and use this information to decide what to do next (either locally or globally). See Rios and Sahinidis (2013) for a review of derivative-free global optimization methods. When the model is complex enough, this is called a meta-model or surrogate-function or response surface approach. Crucially, the model could be a point estimate of $f$ (e.g., the fit of a radial basis network function to our observations), or we could be Bayesian and somehow get a full posterior distribution over $f$ (e.g., via a Gaussian process).

Bayesian optimization uses the posterior over $f$ (in particular, the joint conditional posterior mean and variance at any point) to guide the search of the (global) optimum via some principled heuristic. The classical choice is to maximize the expected improvement over the current best point, but there are even fancier methods, like minimizing the expected entropy over the location of the minimum.

The empirical result here is that having access to a posterior, even if partially misspecified, generally produces better results than other methods.  (There are caveats and situations in which Bayesian optimization is no better than random search, such as in high dimensions.)

Since you asked -- this has a much higher computational cost than other non-Bayesian methods, and you were wondering why we should be Bayesian. The assumption here is that the cost involved in evaluating the true $f$ (e.g., in a real scenario, a complex engineering or machine learning experiment) is much larger than the computational cost for the Bayesian analysis, so being Bayesian pays off.

What can we learn from this example?

First, why does Bayesian optimization work at all? I guess that the model is wrong, but not that wrong, and as usual wrongness depends on what your model is for. For example, the exact shape of $f$ is not relevant for optimization, since we could be optimizing any monotonic transformation thereof. I guess nature is full of such invariances. So, the search we are doing might not be optimal (i.e., we are throwing away good information), but still better than with no uncertainty information.

Second, our example highlights that it is possible that the usefulness of being Bayesian or not depends on the context, e.g. the relative cost and amount of available (computational) resources. (Of course if you are a hardcore Bayesian you believe that every computation is Bayesian inference under some prior and/or approximation.)

Finally, the big question is -- why are the models we use not-so-bad after all, in the sense that the posteriors are still useful and not statistical garbage? If we take the No Free Lunch theorem, apparently we shouldn't be able to say much, but luckily we do not live in a world of completely random (or adversarially chosen) functions.

More in general, since you put the "philosophical" tag... I guess we are entering the realm of the problem of induction, or the unreasonable effectiveness of mathematics in the statistical sciences (specifically, of our mathematical intuition & ability to specify models that work in practice) -- in the sense that from a purely a priori standpoint there is no reason why our guesses should be good or have any guarantee (and for sure you can build mathematical counterexamples in which things go awry), but they turn out to work well in practice.