How to understand the posterior distribution is the same as likelihood function

Question

So I read this post Why is the posterior distribution the same as the likelihood function when uniform prior distribution is used in Bayesian Analysis, and learned that when we have a uniform prior, the posterior distribution is the same as the likelihood function. However, we also have that What is the reason that a likelihood function is not a pdf, i.e. likelihood is not a pdf. For example, the sum of the likelihood might not be 1.

Given these, how should we understand the posterior distribution is the same as the likelihood function when we have a uniform prior? Does this mean the sum of a posterior distribution doesn't need to be 1?

Answer 1

When a Bayesian posterior is derived from a likelihood function that does not integrate (or sum) to unity the posterior function is simply re-scaled to make that integral (or sum) equal one in order that the posterior can be a 'proper' probability distribution.

The use of a uniform prior makes the scaled likelihood function into the posterior and so it might make one wonder whether such a Bayesian approach offers anything beyond a pure likelihood approach. See these little books if you are interested in likelihood-based inference:

• Likelihood by A.W.F. Edwards
• Statistical Evidence, A Likelihood Paradigm by Richard Royall.

Answer 2

The likelihood function is defined up to a multiplicative constant. For instance, the likelihood associated with an ordered sample of size n is n! times the likelihood associated with the unordered sample. More generaly, the likelihood associated with a sufficient statistic is proportional to the likelihood associated with the original sample.

Furthermore, the likelihood function is parameterisation invariant in that changing the parameterisation of the model does not change the likelihood function, in the sense that $\ell_\eta(\eta_0)=\ell_\theta(\theta_0)$ if $\eta=F(\theta)$ and $\eta_0=F(\theta_0)$. This means that the likelihood function does not obey the change-of-variable rule for density functions, i.e., no Jacobian appears in the transform. Therefore, this is a further argument for likelihoods not being pdfs.

A formal equality between a likelihood and a posterior density thus does not signify much, as it depends on arbitrary choices of parameterisation and statistics.