Bayesian neural networks and out-of-distribution data?

Question

In a Bayesian neural network (for classification) the posterior predictive distribution is $$ P(y=c \mid {\bf x}, \mathcal D_{train}) = \int P(y=c \mid {\bf x}, \theta) p(\theta \mid \mathcal D_{train}) d\theta $$

Let's assume that we have enough training data $\mathcal D_{train}$ such that if $\bf x$ is quite similar to the training data the uncertainty in the posterior (predictive) prediction is low. So, we assume that in this region we have low aleatoric uncertainty and due to the amount of training data the weights distribution is not quite different from a delta function.

Can we say something about the posterior (predictive) prediction uncertainty if $\bf x$ is OOD (out-of-distribution)? In the literature it is often argued like this: " consistent in the region of training data, and becomes increasingly diverse when the input $\bf x$ is far from the training data" (Source: Predictive Uncertainty Estimation via Prior Networks). However, is there a general theoretic argument (or proof) for arbitrary models (here Neural Networks) which underpins that?

Gaussian processes for instance are designed that this nice property holds. As simple counter example, let's consider univariate linear regression with known slope. The only random variable of the model is the intersect. Here, the uncertainity of the predicted $y$ is the same for OOD and in-distribution $x$.

So, is there an argument that Bayesian neural networks can handle the uncertainty of $OOD$ appropriate?

Answer 1

For every model how well does it deal with making predictions on unseen data will depend on two factors, on how much does the training data resemble the new data and on the assumptions made by the model. In the case of simple models like linear regression, it usually is easy to reason about how the model would behave for the new data because the model structure is fairly straightforward. The same applies to the uncertainty estimates, they will depend on the data, the likelihood, and the priors.

Neural networks are so flexible because they are easily able to approximate very complicated functions. Unfortunately, for the same reason, they are hard to interpret and are generally considered as black-box-ish algorithms. More than this, they can easily learn some strange, complicated functions and overfit. The same applies to Bayesian neural networks, the only difference is that they additionally give you the uncertainty estimates for the parameters. What follows, you usually have no guarantees whatsoever on how well your data approximate the distribution of the data (if it won't overfit) and what the uncertainty estimates would be worth. Moreover, the results of any Bayesian model depend on the priors, and with Bayesian neural networks we usually have no way of choosing reasonable, informative priors and we default to generic ones.

To make this even more complicated, Bayesian neural networks are usually too big to sample from them using MCMC, so instead, you are forced to approximate the posterior distribution (e.g. variational approximation). Those approximations may, or may not, be good approximations of the posterior distribution. So you end up with having a model approximating the distribution of the data, and having results that approximate the posterior distribution.

TL;DR you are making hell a lot of assumptions that will affect the results and validity of the uncertainty estimates.