Why does the Bayesian posterior concentrate around the minimiser of KL divergence? Consider the Bayesian posterior $\theta\mid X$. Asymptotically, its maximum occurs at the MLE estimate $\hat \theta$, which just maximizes the likelihood $\operatorname{argmin}_\theta\, f_\theta(X)$.
All these concepts—Bayesian priors, maximizing the likelihood—sound super principled and not at all arbitrary. There’s not a log in sight.
Yet MLE minimizes the KL divergence between the real distribution $\tilde f$ and $f_\theta(x)$, i.e., it minimizes
$$ KL(\tilde f \parallel f_\theta)  = \int_{-\infty}^{+\infty} \tilde f(x) \left[ \log \tilde f(x) - \log f_\theta(x) \right] \, dx $$
Woah—where did these logs come from? Why KL divergence in particular?
Why, for example, does minimizing a different divergence not correspond to the super principled and motivated concepts of Bayesian posteriors and maximizing likelihood above?
There seems to be something special about KL divergence and/or logs in this context. Of course, we can throw our hands in the air and say that’s just how the math is. But I suspect there might be some deeper intuition or connections to uncover.
 A: Use of logarithms in calculations like this comes from information theory.  In the particular case of the KL divergence, the measure can be interpreted as the relative information of two distributions:
$$\begin{equation} \begin{aligned}
KL(\tilde{f} \parallel f_\theta) 
&= \int \limits_{-\infty}^\infty \tilde{f}(x) (\log \tilde{f}(x) - \log f_\theta (x)) \ dx \\[6pt]
&= \Bigg( \underbrace{- \int \limits_{-\infty}^\infty \tilde{f}(x) \log f_\theta(x) \ dx}_{H(\tilde{f}, f_\theta)} \Bigg)  - \Bigg( \underbrace{- \int \limits_{-\infty}^\infty \tilde{f}(x) \log \tilde{f}(x) \ dx}_{H(\tilde{f})} \Bigg), \\[6pt]
\end{aligned} \end{equation}$$
where $H(\tilde{f})$ is the entropy of $\tilde{f}$ and $H(\tilde{f}, f_\theta)$ is the cross-entropy of the $\tilde{f}$ and $f_\theta$.  Entropy can be regarded as measures of the average rate of produced by a density (thought cross-entropy is a bit more complicated).  Minimising the KL divergence for a fixed value $\tilde{f}$ (as in the problem you mention) is equivalent to minimising the cross-entropy, and so this optimisation can be given an information-theoretic interpretation.
It is not possible for me to give a good account of information theory, and the properties of information measures, in a short post.  However, I would recommend having a look at the field, as it has close connections to statistics.  Many statistical measures involving integrals and sums over logarithms of densities are simple combinations of standard information measures used in measure theory, and in such cases, they can be given interpretations in terms of the underlying levels of information in various densities, etc.
