# Marginal Likelihood as probability distribution

I understand that Likelihood is not a probability distribution but what about Marginal likelihood ? Suppose we are given a set of data points $\pmb{X}$ such that each data point $x_{i} \sim p(x_{i}|\theta)$. Now $\theta$ is also a random variable such that $\theta \sim p(\theta|\alpha)$. Then Wikipedia describes marginal likelihood as the probability $p(\pmb{X}|\alpha)$ where $\alpha$ is integrated out(marginalized). Does this mean that all marginal likelihoods integrate to 1 and therefore formally are probability distributions as well ? Or is it just an error in writing the marginal likelihood as $p(\pmb{X}|\alpha)$ for a lack of a better alternative. Also various texts refer to the likelihood as $p(\pmb{X}|\theta)$ , are all of them being non-rigorous ?

• Likelihood is a function of the (fixed but unknown) parameters given the observed data, so it's not a probability distribution. Integrating out some of those parameters doesn't make it one. – jbowman Dec 13 '17 at 3:15

Suppose we have the following joint probability density: $$p(x,\theta,\alpha) %= p(x|\theta)\,p(\theta|\alpha)\,p(\alpha) ,$$ where $x = (x_1, \ldots, x_n)$. Because this is a probability density, we have $$\iiint p(x,\theta,\alpha)\,dx\,d\theta\,d\alpha = 1 .$$ Now suppose the joint density factors into the following three densities (two conditional densities and one marginal density): $$p(x,\theta,\alpha) = p(x|\theta)\,p(\theta|\alpha)\,p(\alpha) .$$ Given this factorization, we know \begin{align} \int p(x|\theta)\,dx &= 1 \\ \int p(\theta|\alpha)\,d\theta &= 1 \\ \int p(\alpha)\,d\alpha &= 1 . \end{align} Note that $p(x|\theta)$ is the likelihood for $\theta$ and $p(\theta|\alpha)$ is the likelihood for $\alpha$. As such, there is no requirement that either integrate to one or to any finite value for that matter: \begin{align} \int p(x|\theta)\,d\theta &= {}? \\ \int p(\theta|\alpha)\,d\alpha &= {}? . \end{align}

If we integrate out $\theta$ (not $\alpha$), we obtain $$p(x|\alpha) = \int p(x|\theta)\,p(\theta|\alpha)\,d\theta ,$$ where $$\int p(x|\alpha)\,dx = 1 .$$ Note that $p(x|\alpha)$ is the marginal likelihood for $\alpha$. Again there is no requirement that it integrate to one or any finite value: $$\int p(x|\alpha)\,d\alpha = {}? .$$

• $p(x|\theta)$ is the likelihood for $\theta$ when $x$ is observed and $\theta$ is a random variable and the same symbol represents the conditional probability when $x$ is a Random variable and $\theta$ is given ? So there's a lot of overlapping symbology going on here and it's just a matter of being at loss with more symbolical descriptions ? –  redenzione11 Dec 13 '17 at 9:46
• If we fix the value of $\theta$, then $p(x|\theta)$ is the conditional density for $x$. If we fix a value for $x$, then $p(x|\theta)$ is the likelihood for $\theta$. – mef Dec 13 '17 at 13:08