Dropping the normalization constant in Bayesian inference [duplicate]

Question

Could anyone please explain to me (or provide a situation) where one would drop the normalising constant (or marginal probability term) from Bayes rule when performing Bayesian Inference?

New to Bayesian theory so any help appreciated.

Thanks in advance.

Answer 1

I can think of a couple situations where this would be OK:

1. You want to get the maximum a-posteriori (MAP) estimate of your parameter. In that case, the normalizing constant will not affect the value of the estimate.
2. You are using MCMC or acceptance rejection to estimate the posterior.

For the first case, note that if we are performing inference on $\theta$ using data $\mathbf{x}$, then:

$$P(\theta|\mathbf{x}) \propto P(\theta)P(\mathbf{x}|\theta)$$

The MAP estimate of $\theta$ is the value of $\theta$ that maximizes $P(\theta|\mathbf{x})$. However, as you may remember from calculus, if $x^*$ maximizes a function $f(x)$, then it also maximizes $cf(x)$, where $c>0$. In this case,

$$f:=f(\theta)=P(\theta|\mathbf{x})=c P(\theta)P(\mathbf{x}|\theta)$$

for some normalizing constance $c$. Therefore, if you find $\theta^*$ which maximizes $P(\theta)P(\mathbf{x}|\theta)$, you will have also found the value that would have maximized the actual posterior.

For the second example, I will simply state that these methods were designed to avoid the ugly integration in the denominator using advanced numerical approaches. I won't go int MCMC, but rejection methods work by creating an "envelope" above $P(\theta)P(\mathbf{x}|\theta)$ that can be used to generate draws from the target distribution. Here are some details.