I came across a question 8 at the end of chapter 3 of the book:
"Give two simple examples showing a case in which a prior distribution would not be overwhelmed by data, regardless of the sample size"
Can anyone provide any examples where this would be the case, because I thought that as the number of data points goes to infinity, the data will always overwhelm the prior (assuming no ill-defined priors).
One way is to set a prior that is a constant. For example, take a simple linear regression context where you have an intercept, one slope, and an error term. If you set the prior on beta to be $\beta \sim \text{N}(5, 0)$, then no amount of data can overwhelm that prior. You are multiplying an arbitrarily large number of data points to something that has no variance; you will get 5, no matter the sample size.
Another example would be lack of identification.
Assume a proper prior $\pi(\theta)$. We obtain that the posterior is equal to the prior, $\pi(\theta|y)=\pi(\theta)$, if $f(y|\theta)$ does not depend on $\theta$, i.e., if the likelihood is not informative about the parameter of interest:
\begin{eqnarray*} \pi(\theta|y)&=&\frac{f(y|\theta)\pi(\theta)}{\int f(y|\theta)\pi(\theta)d\theta}\\ &=&\frac{f(y|\theta)\pi(\theta)}{f(y|\theta)\int \pi(\theta)d\theta}\\ &=&\frac{\pi(\theta)}{\int \pi(\theta)d\theta}\\ &=&\pi(\theta) \end{eqnarray*}
Since the likelihood does not depend on $\theta$, the data does not modify our beliefs about $\theta$, so that the posterior still is equal to the prior.
