For numerical Bayesian inference we have Posterior~Prior*Likelihood. In MCMC we do not need to calculate the denominator in Bayes rule. My question is that can I multiply the Likelihood by a large constant while making sure that the posterior is still the same?
$\begingroup$
$\endgroup$
0
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
0
When you multiply the likelihood by the prior, the resulting function may no longer integrate to $1$, hence why you need to know the normalising constant to analytically solve the posterior.
MCMC samples randomly with frequency proportional to the posterior density (or height of the distribution at any point), without actually knowing it. Hence why you don't need to know the normalising constant when emperically estimating the posterior by MCMC.
So theoretically, multiplying the likelihood by some constant should not affect MCMC. However, you might run into computational problems or numerical instability.
answered Aug 12, 2018 at 21:26
$\begingroup$
$\endgroup$
Not only is that allowed, but the multiple you get is still a valid likelihood function. A valid likelihood function is defined by the requirement that $L_\mathbf{x}(\theta) \propto p(\mathbf{x}|\theta)$ (i.e., it is proportional to the sampling density, with respect to the parameter vector). Multiplication of a likelihood function by a positive value that does not depend on the parameter vector leaves this proportionality requirement intact, so the result is another valid likelihood function. While we commonly refer to "the" likelihood function, this is actually a class of functions defined up to a positive multiplicative constant.
MCMC methods that use the likelihood function (as opposed to the sampling density) should be set up propoerly to allow any instance of the likelihood function, and so multiplication by a constant does not matter in this case. Moreover, multiplication by a constant does not even change the fact that you are still using a valid likelihood function.
