1. What do we mean by "Integrating out the parameters" in Marginal likelihood? Particularly in the posterior formula.

The marginal likelihood in a posterior formulation, i.e P(theta|data) , as per my understanding is the probability of all data without taking the 'theta' into account. So does this mean that we are integrating out theta? If that is the case, do we apply limits over the integral in that case? What are those limits?

  1. Some people say that we can integrate the product of prior and likelihood to obtain the marginal likelihood. So how does this links in posterior formulation, which already has this product in numerator.?

  2. Calculation of Marginal likelihood is ignored at times. Its a constant, that is not used usually. However, we don't call such approaches "Fully Bayesian". What kind of advantages does Full Bayesian provide if you are adding a constant.


Considering a statistical model represented by a family of probability densities (wrt the same measure $\text dx$) $$\mathfrak F=\{f(\cdot;\theta)\,;\ \theta\in\Theta\}$$ each density within that family is associated with a parameter $\theta$.

Given in addition a probability density $\pi(\cdot)$ over $\Theta$ (endowed with a measure $\text d\theta$) the marginal likelihood is defined as $$m(x) = \int_{\Theta} f(x;\theta)\,\pi(\theta)\text d\theta\tag{1}$$ and is therefore

  1. the marginal density of $X$ associated with the joint density $f(x;\theta)\,\pi(\theta)$ of the pair $(X,\theta)$
  2. free of the parameter $\theta$, which is effectively integrated out in (1)
  3. a misnomer in that it is a function of $x$ and only of $x$, while the likelihood is primarily a function of $\theta$, which is why the term evidence or integrated likelihood is often preferred
  4. an average of all possible density values at $x$, when weighting each value of $\theta$ with $\pi(\theta)$
  5. the prior predictive density resulting from (i) generating $\theta$ from the prior and (ii) $X$ from the model associated with the generated value of $\theta$
  6. the normalising constant in the posterior distribution, so that it integrates to one in $\theta$
  7. a measure of how the model $\mathfrak F$ fits the realisation $x$ of $X$ and hence a tool for model comparison (via Bayes factors)
  8. not necessarily needed when approximating Bayesian procedures by simulation (although this approach remains "fully Bayesian") since simulation techniques often bypass the normalising constant.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.