I'm trying to get my head around Bayesian inference and the difference between the posterior and likelihood. Going off the back of these answers, I'm under the impression that the posterior is $P(data|model)$ whereas the likelihood is $L(model|data)$.

However, I'm a structural biologist. The landmark paper bringing Bayesian inference to the field can be found here

Quoting from the paper:

The posterior density

$p(X|D,I) \propto p(D|X,I)p(X|I)$

factorizes into two natural components: The likelihood function $p(D|X, I)$ combines a forward model and an error distribution and quantifies the likelihood of observing data D given a molecular structure X ... The prior density $p(X|I)$ takes prior knowledge about biomolecular structures into account and is determined by the physical energy and the temperature of the system.

So here the likelihood is the data given the model. Is it okay to switch the model and data around like this?


Generally we discuss likelihoods for parameters of a model. In your case, the parameter is the molecular structure. Below I will use the term parameter just to be more general, but you can replace the term "parameter" with "molecular structure" for this particular example. Similarly, in keeping with standard statistical notation, I will use $\theta$ to correspond to a parameter, but you can replace $\theta$ with $X$.

A statistical (or probability) model is a generative model for your data given a set of assumptions including the value of the parameter. Thus, $$ p(D|\theta,I) $$ is your statistical model which depends on the unknown parameter $\theta$ and the known other information $I$, e.g. temperature of the system. If $D$ is discrete (continuous), then $p(D|\theta,I)$ is a probability mass (density) function.

Bayes' Rule allows us to switch the data and parameter around to obtain the posterior $p(\theta|D,I)$ based on the model and the prior $p(\theta|I)$, i.e.
$$ p(\theta|D,I) \propto p(D|\theta,I)p(\theta|I)$$ where the proportionality sign just drops the constant that makes this posterior a valid probability mass or density function depend on whether $\theta$ is discrete or continuous.

You can safely stop here with no mention of likelihood and this is what I often do when teaching introductory Bayesian statistics.

The likelihood function is the probability mass or density function but thought of as a function of the unknown parameters for a particular observation $D$, i.e. $$ L(\theta|D) = p(D|\theta,I). $$ It is not a probability mass or density function.

The only difference between the model and the likelihood is our perspective. The model says "if we knew the value of the parameter, then we would know the probability distribution for our data". But, of course, the whole point is that we don't know the value of the parameter. The likelihood says "these values of the parameter make our data more likely and therefore it is more likely that these are the true values of the parameter."

  • $\begingroup$ so if this model (and it's associated likelihood) doesn't actually consider the prior information, why is it written in ? should it not be $p(\theta|D,I) \propto p(D|\theta)p(\theta|I)$ $\endgroup$ May 27 '16 at 13:30
  • $\begingroup$ If $D$ is conditionally independent of $\theta$ given $I$ then $p(D|\theta)=p(D|\theta,I)$. This is very problem specific and I don't know enough about what $I$, $D$, and $\theta$ represent and the science of your problem to determine whether conditional independence is a reasonable assumption. $\endgroup$
    – jaradniemi
    May 27 '16 at 14:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.