I came across a question 8 at the end of chapter 3 of the book:

"Give two simple examples showing a case in which a prior distribution would not be overwhelmed by data, regardless of the sample size"

Can anyone provide any examples where this would be the case, because I thought that as the number of data points goes to infinity, the data will always overwhelm the prior (assuming no ill-defined priors).

  • 1
    $\begingroup$ Events of null prior probability are also of null posterior probability, no matter what is the size of the sample. $\endgroup$ – Olivier Jan 13 '18 at 23:20
  • $\begingroup$ Related stats.stackexchange.com/questions/200982/… $\endgroup$ – Tim Mar 2 '18 at 9:20

One way is to set a prior that is a constant. For example, take a simple linear regression context where you have an intercept, one slope, and an error term. If you set the prior on beta to be $\beta \sim \text{N}(5, 0)$, then no amount of data can overwhelm that prior. You are multiplying an arbitrarily large number of data points to something that has no variance; you will get 5, no matter the sample size.

  • $\begingroup$ Thanks a lot, this actually makes sense now. I guess this also nests Olivier's answer: all other values of $\beta$ would have in this case prior probability of zero, hence any number returned by the likelihood function multiplied by zero is zero, except at the parameter value of 5. However, Mark White, would not the answer be whatever the likelihood function evaluates to at 5 (ignoring normalization of prior $\times$ likelihood)? Let's say the likelihood of $\beta=5$ is 2. Hence, 2 multiplied by 1 (as prior probability at 5 is 1), giving unnormalized posterior probability of 2. $\endgroup$ – AlexMe Jan 14 '18 at 2:37
  • $\begingroup$ so basically any data distribution when multiplied by a distribution with zero variance will result in having fixed prior irrespective of the data taken. Right ? $\endgroup$ – Upendra Pratap Singh Jan 16 '18 at 9:42

Another example would be lack of identification.

Assume a proper prior $\pi(\theta)$. We obtain that the posterior is equal to the prior, $\pi(\theta|y)=\pi(\theta)$, if $f(y|\theta)$ does not depend on $\theta$, i.e., if the likelihood is not informative about the parameter of interest:

\begin{eqnarray*} \pi(\theta|y)&=&\frac{f(y|\theta)\pi(\theta)}{\int f(y|\theta)\pi(\theta)d\theta}\\ &=&\frac{f(y|\theta)\pi(\theta)}{f(y|\theta)\int \pi(\theta)d\theta}\\ &=&\frac{\pi(\theta)}{\int \pi(\theta)d\theta}\\ &=&\pi(\theta) \end{eqnarray*}

Since the likelihood does not depend on $\theta$, the data does not modify our beliefs about $\theta$, so that the posterior still is equal to the prior.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.