Could anyone please explain to me (or provide a situation) where one would drop the normalising constant (or marginal probability term) from Bayes rule when performing Bayesian Inference?

New to Bayesian theory so any help appreciated.

Thanks in advance.


I can think of a couple situations where this would be OK:

  1. You want to get the maximum a-posteriori (MAP) estimate of your parameter. In that case, the normalizing constant will not affect the value of the estimate.
  2. You are using MCMC or acceptance rejection to estimate the posterior.

For the first case, note that if we are performing inference on $\theta$ using data $\mathbf{x}$, then:

$$P(\theta|\mathbf{x}) \propto P(\theta)P(\mathbf{x}|\theta)$$

The MAP estimate of $\theta$ is the value of $\theta$ that maximizes $P(\theta|\mathbf{x})$. However, as you may remember from calculus, if $x^*$ maximizes a function $f(x)$, then it also maximizes $cf(x)$, where $c>0$. In this case,

$$f:=f(\theta)=P(\theta|\mathbf{x})=c P(\theta)P(\mathbf{x}|\theta)$$

for some normalizing constance $c$. Therefore, if you find $\theta^*$ which maximizes $P(\theta)P(\mathbf{x}|\theta)$, you will have also found the value that would have maximized the actual posterior.

For the second example, I will simply state that these methods were designed to avoid the ugly integration in the denominator using advanced numerical approaches. I won't go int MCMC, but rejection methods work by creating an "envelope" above $P(\theta)P(\mathbf{x}|\theta)$ that can be used to generate draws from the target distribution. Here are some details.

  • $\begingroup$ Great, thank you! Could you possibly give me some brief reasons as to why it would be ok in these situations? Just to help my understanding. Thanks ! $\endgroup$ – Maeve90 May 7 '16 at 7:46
  • $\begingroup$ @MaeveRyan sure thing. Added some explanation. $\endgroup$ – user75138 May 7 '16 at 20:36
  • $\begingroup$ @MaeveRyan glad to help. Please mark this answer as accepted if it has addressed your question. That way it doesn't appear that you are still awaiting further answers. $\endgroup$ – user75138 May 7 '16 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.