# Under what conditions will a Bayesian posterior fail to converge to a point mass?

Let's say you have a Bayesian model:

$$\theta' \sim g(\theta|\mu)$$ $$y \sim p(y|\theta')$$

And we have some data ($n$ data points) $\mathbf{y}_n$, which we will use to perform inference on $\theta'$ by calculating $p(\theta'|\mathbf{y}_n)$

My interpretation of the posterior $p(\theta'=x|\mathbb{y}_n)$ in this case is as follows:

Given the proposed model and observations $\mathbf{y}_n$, what is the probability that the value of the generated $\theta'$ was x

Note: This is not the same as likelihood, which says "how probable is the data given $\theta'$...I am explicitly taking into account the probability of both the generated model $(\theta' = x)$ and the observations $\mathbf{y}_n$ since both are modeled as random.

Now, if the model were correct, then as I collect more data ($n \to \infty$), I'd expect the likelihood function $p(\theta'|\mathbf{y}_n)$ to concentrate more and more around a particular value of $\theta'$. Let's call this a convergent posterior.

However, most modern statisticians would probably say a model is merely a device or approximation to the true statistical model of the system under study -- this approximate model is used to provide a simplified context within which we can interpret some aspects of the system (e.g., if $\theta'$ represents some "mean response time" that we wish to model as constant for a given system...it may not be, but we want to see how much we can explain if we assume it is).

So, what about the much more likely case that our model is wrong -- what happens as we collect more data in this (highly likely) case?

I can imagine two broad scenarios:

1. In the context of our model, the data behave as if they were generated by a single (or very small interval) of $\theta'$.
2. The data behave as if $\theta'$ were randomly chosen for each $y_i$ from some mixture distribution over $\theta'$.

(1) is what most of us expect as we collect more data and should yield at least an approximately convergent posterior.

However, in (2) the data $\mathbf{y}_n$ appear to be generated from a mixture of $\theta'$ values, which I would expect would be shown as a non-convergent posterior for $\theta'$ (e.g., it converges to some uni-modal or multi-modal distribution).

In fact, in the extreme case, wouldn't a poorly specified model allow the possibility that the 95% posterior credible interval becomes wider than the 95% prior credible interval (since the likelihood will not "settle down" to a high-likelihood $\theta'$)?! The most pathological case would be where your model is so inconsistent with the data that all credible intervals for $\theta'$ diverge as you collect more data (i.e., you likelihood model is constantly "surprised" by new observations).

My Question:

It seems to me that from a modeling perspective, (2) should be the norm and not the exception, since the data does not actually follow our model, therefore:

What are necessary or sufficient conditions under which (2) is possible? (References to papers are also appreciated).

Here is a paper I found, which suggests my intuition about (2) is correct.