How posterior function is calculated in JAGS

Question

I have a theoretical question. I understand the JAGS samples from the posterior function of a model. But I don't understand (nor I can find in the documentation) how it calculates the posterior in the first place (the function from which it later samples from using Gibbs).

Answer 1

The clever thing with the various Markov-Chain Monte-Carlo (MCMC) samplers (like JAGS, WinBUGS, Stan, pymc3 etc.) is that they do not need to calculate the posterior itself. They only need the (log density function of the) prior distribution and the (log-)likelihood. The product of these two (or the sum, when working on the log-scale) is only proportional to the posterior (we lack a normalizing constant). Being able to work with that may not sound like a huge bit of progress, but actually helps a lot, because we can usually not find the necessary normalizing constant analytically. In contrast, it is often pretty easy to write down the log-likelihood and the log density of the prior distribution. That (either conditional on some parameters for Gibbs sampling, or unconditionally for e.g. Metropolis-Hastings or Hamiltonian Monte Carlo) is enough to put MCMC samplers to work.

Answer 2

There's only one way of obtaining posterior distribution: by applying Bayes theorem. If your likelihood is $f(X|\theta)$ and the prior is $g(\theta)$, then the posterior is

$$ g(\theta|X) \propto f(X|\theta)\, g(\theta) $$

where the normalizing constant $f(X)$ is ignored, because it is not needed for MCMC, or optimization.

For example, if you assume that $X$ is distributed according to binomial distribution with known $n$ and unknown $p$, and you assume uniform prior for $p$. So if you want to calculate posterior probability of observing some particular value of $\theta$, you multiply binomial portability mass function evaluated at your data point $x$, with parameters $n$ and $p$, and multiply it with uniform probability density function evaluated at $\theta$. No black magic involved.