I read a section on Bayesian sensitivity analysis in the following book by Carlin and Louis (2009), 'Bayesian Methods for Data Analysis' (3rd ed.), CRC Press.

The context is a sensitivity analysis of a Bayesian model that lead to posterior $p(\theta|y)$. In the sensitivity analysis the likelihood specification is changed or a different prior is chosen. The new posterior is called $p_{NEW}( \theta| \bf{y})$. Assume we can only use MCMC to sample from both posteriors. Then after obtaining the posterior $p$, one would need to run the MCMC sampler again to obtain $p_{NEW}$.

On p. 182 the authors describe an alternative method. It usse the MCMC samples from a converged sampler to invoke an importance sampler to use for sampling from $p_{NEW}$. That means that instead of changing the prior and running the sampler again, an importance sampler is used with the available MCMC samples as importance sampling density.

However I cannot follow the argument made there, see my question below. So we have $p(\theta|y)$ and given a NEW prior or a change in the likelihood we arrive at the NEW posterior $p_{NEW}( \theta| \bf{y})$. Then the authors go on as follows (p. 182):

'Fortunately, a little algebraic work eliminates the need for further sampling. Suppose we have a sample $\{\theta_1,...\theta_N\}$ from a posterior $p(\theta|y)$, which arises from a likelihood $f(\bf{y}|\theta)$ $= \prod_i f(y_i|\theta)$ and a prior $\pi(\theta)$. To study the impact of deleting case $k$, we see that the new posterior is $$p_{NEW}( \theta| y) \propto \frac{f(y|\theta) \pi(\theta)}{f(y_k|\theta)} \propto \frac{1}{f(y_k|\theta)} p(\theta | y) $$ In the notation of Subsection 3.3.2, we can use $g(\theta) = p(\theta|y)$ as an importance sampling density, so that the weight function is given by $$w(\theta) = \frac{p_{NEW}( \theta| y) }{p(\theta | y) } = \frac{1}{f(y_k|\theta)} $$ Thus for any posterior function of interest $h(\theta)$, we have $\hat{E}(h(\theta)|y) = \sum_j h(\theta_j)/N$, and $$\hat{E}_{NEW}(h(\theta)|y) = \frac{\sum_j h(\theta_j)w(\theta_j)}{\sum_j w(\theta_j)}$$ [...]. Because $p$ and $p_{NEW}$ should be reasonably similar the former should be a good importance sampling density for the latter and hence the approximation in [the last] equation [above] should be good for moderate N.'

I cannot follow what is the relation of a change in likelihood or prior to arrive at $p_{NEW}$ to the conjecture about the omitted case $k$. I can follow the math but how does this help me when I want to use the prior $\pi_{NEW}(\theta)$ instead of $\pi(\theta)$ and I have $\{\theta_1,...\theta_N\}$, for example?

  • 1
    $\begingroup$ The only change between $p_{NEW}$ and $p$ is the removal of the term $f(y_k|theta)$ in the likelihood. It thus makes sense that the importance weight is $1/f(y_k|\theta)$. If instead you switch from prior $\pi$ to prior $\pi_{NEW}$ the weight should be $\pi_{NEW}(\theta)/\pi(\theta)$. $\endgroup$
    – Xi'an
    Oct 27 '20 at 19:33
  • $\begingroup$ @Xi'an Thanks. I still think it is somewhat strange to bring up the example with $k$ instead of doing it for a new prior. So this should also work with new likelihood $f_{NEW}$ which would lead to weights $f_{NEW}/f$ right? $\endgroup$
    – tomka
    Oct 28 '20 at 9:06
  • $\begingroup$ @Xi'an If that is so, it is not mentioned there. It would be a plot hole. $\endgroup$
    – tomka
    Oct 28 '20 at 10:12
  • $\begingroup$ @Xi'an yes but it comes totally at surprise at that point imho, it's out of context. The paragraph above is only about changing the likelihood or the prior, not at all about cross validation or leaving out cases. I do understand it at best in the context of leaving out an 'outlier' (influential observation) and study the impact of that. $\endgroup$
    – tomka
    Oct 28 '20 at 10:21
  • $\begingroup$ anyways, thanks for clarifying :) $\endgroup$
    – tomka
    Oct 28 '20 at 10:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.