Marginal Likelihood as probability distribution I understand that Likelihood is not a probability distribution but what about Marginal likelihood ? Suppose we are given a set of data points $\pmb{X}$ such that each data point $x_{i} \sim p(x_{i}|\theta)$. Now $\theta$ is also a random variable such that $\theta \sim p(\theta|\alpha)$. Then Wikipedia describes marginal likelihood as the probability $p(\pmb{X}|\alpha)$ where $\alpha$ is integrated out(marginalized). Does this mean that all marginal likelihoods integrate to 1 and therefore formally are probability distributions as well ? Or is it just an error in writing the marginal likelihood as $p(\pmb{X}|\alpha)$ for a lack of a better alternative. Also various texts refer to the likelihood as $p(\pmb{X}|\theta)$ , are all of them being non-rigorous ?
 A: Suppose we have the following joint probability density:
\begin{equation}
p(x,\theta,\alpha) %= p(x|\theta)\,p(\theta|\alpha)\,p(\alpha) , 
\end{equation}
where $x = (x_1, \ldots, x_n)$. Because this is a probability density, we have
\begin{equation}
\iiint p(x,\theta,\alpha)\,dx\,d\theta\,d\alpha = 1 .
\end{equation}
Now suppose the joint density factors into the following three densities (two conditional densities and one marginal density):
\begin{equation}
p(x,\theta,\alpha) = p(x|\theta)\,p(\theta|\alpha)\,p(\alpha) .
\end{equation}
Given this factorization, we know
\begin{align}
\int p(x|\theta)\,dx &= 1 \\
\int p(\theta|\alpha)\,d\theta &= 1 \\
\int p(\alpha)\,d\alpha &= 1 . 
\end{align}
Note that $p(x|\theta)$ is the likelihood for $\theta$ and $p(\theta|\alpha)$ is the likelihood for $\alpha$. As such, there is no requirement that either integrate to one or to any finite value for that matter:
\begin{align}
\int p(x|\theta)\,d\theta &= {}? \\
\int p(\theta|\alpha)\,d\alpha &= {}? .
\end{align}
If we integrate out $\theta$ (not $\alpha$), we obtain
\begin{equation}
p(x|\alpha) = \int p(x|\theta)\,p(\theta|\alpha)\,d\theta ,
\end{equation}
where
\begin{equation}
\int p(x|\alpha)\,dx = 1 . 
\end{equation}
Note that $p(x|\alpha)$ is the marginal likelihood for $\alpha$. Again there is no requirement that it integrate to one or any finite value:
\begin{equation}
\int p(x|\alpha)\,d\alpha = {}? . 
\end{equation}
