I came across this answer which states that:

NOT all the MCMC methods avoid the need for the normalising constant.

I was under the impression that one of the strengths of the MCMC methods (usually employed for parameter inference) is that they avoid the need for obtaining the Bayesian evidence, only requiring evaluating the likelihood and the prior.

Apparently this is not true, so my question is: what MCMC methods need to calculate the Bayesian evidence (normalising constant)?

  • $\begingroup$ There is a huge literature on using MCMC methods to calculate the normalising constant (evidence) and thus this question is too broad. $\endgroup$
    – jaradniemi
    Nov 9 '18 at 14:55
  • 1
    $\begingroup$ The question is not really about how to calculate the normalizing constant but which MCMC methods employed in parameter inference do it and why. $\endgroup$
    – Gabriel
    Nov 9 '18 at 14:59
  • $\begingroup$ @Gabriel (sorry accidentally entered comment before finishing), the evidence function $P(D)=\int_{\Theta} P(D|\theta)P(\theta)$ is a sum of the probability(up to a proportionality constant) of all possible models $\theta$. If you had a way to calculate this, then you wouldn't need to use an MCMC sampler. The answer you posted needs to be reworded. $\endgroup$ Nov 9 '18 at 16:04
  • $\begingroup$ @curious_dan I still don't understand, I haven't posted any answers. $\endgroup$
    – Gabriel
    Nov 9 '18 at 16:10
  • $\begingroup$ Sorry, you said "I came across this answer" and then posted another thread. This is the answer that needs to be reworded. All MCMC samplers avoid the need for the normalizing constant. If you knew the normalizing constant, then you wouldn't use MCMC. $\endgroup$ Nov 9 '18 at 16:13

You need to consider what the actual output from MCMC is: a very large set of samples $\{ \theta_1, \theta_2, \dots, \theta_N\}$ that you hope are representative of the posterior distribution, $\pi(\theta)$, say, of the parameter $\theta $.

MCMC proceeds by starting from some $\theta_0$ moving on to $\theta_1$ and so on until equilibrium is reached. Various indicative tests for convergence exist but proof that it has occurred is not available. It is assumed that after convergence each $\theta_i$ is a draw from the posterior distribution.

You are then in a position to examine the empirical distribution of the MCMC output and summarise it using all the usual tools. You do not need to calculate the evidence to do that.

  • $\begingroup$ Hi, yes I understand all that. But that answer stated that there are in fact MCMC methods that do employ the evidence (normalizing constant), hence my question. If your answer is that there are no MCMC methods that require calculating the evidence to work, could you make that clear? $\endgroup$
    – Gabriel
    Nov 9 '18 at 18:00
  • $\begingroup$ I cannot claim knowledge of every MCMC method that has ever been written - and there are lots. I cannot see why anyone would ever want to calculate the evidence as part of using MCMC, but they might use MCMC to calculate the evidence. $\endgroup$
    – JeremyC
    Nov 9 '18 at 22:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.