Frank Harrell writes in one of our community wikis about the link between "Bayes optimum decision" and the link to probabilistic predictions (and, thus, one of his favorite topics in proper scoring rules for machine learning evaluation).
What is that link?
(And is Bayes optimum decision the same as minimal Bayes risk?)
I can't say what Frank was referring to for sure, but I can do my best to impute what he meant.
Let $\ell(x;\theta)$ be a loss function for a given action $x$. The loss may depend on some parameters $\theta$ which are either known or unknown to us. An important question becomes "For what $x$ is this loss minimized?". If $\theta$ is known and $\ell$ satisfies some regularity conditions, then the question can be answered by "just" applying some minimization techniques.
But what if $\theta$ is unknown? What if it must be estimated from data? In such a case, and assuming a Bayesian framework (a similar approach called Empirical Risk Minimization exists for Frequentists), we need to integrate our loss over our posterior. This requires computation of the following integral
$$ \mathcal{L}(x) = \int_\Theta \ell(x;\theta) p(\theta\vert y) \, d \theta $$
This new function $\mathcal{L}$ is the expected loss over the posterior distribution. So far as I understand, the action which minimizes this quantity is the Bayes optimum decision. In what sense is this the optimal decision? In the sense that it minimizes expected loss. The link between probabilistic predictions (from the posterior distribution) and decision making is now a little clearer.
Back to how this relates to Frank. Frank has previously alluded to this sort of decision making his blog post classification vs. prediction. From that blog post...
Classification is in effect a decision. Optimum decisions require making full use of available data, developing predictions, and applying a loss/utility/cost function to make a decision that, for example, minimizes expected loss or maximizes expected utility
Frank argues (correctly from my pov) that all statistical modelling stops at the output of the posterior density. What we do with that density is then in the domain of decision theory, not statistics. Careful consideration of what each action entails (with respect to loss, what we stand to lose from each action) is required, and classification algorithms à la machine learning ignore those costs to the detriment of whomever is implementing those algorithms, or whomever is subject to the algorithm's decision.
I've written about how this can be done for simple AB tests here. Say we intend to run an AB test to drive click through rate for some website. The loss function might be how much click through we stand to lose if we make the wrong decision (e.g. implement variant A when variant B is truly superior). Clearly, this loss depends on the unknown click through rates. We can use the results of the experiment to estimate these rates, and choose the variant which results in smallest expected loss (where the expectation is taken over posterior distribution of click through rates).
