# Marginal likelihood vs. prior predictive probability

In the Bayesian framework, to me, it seems that the marginal likelihood and the prior predictive distribution/probability are equal. Is that the case? Or maybe this just holds for single data points? Why differ between these two terms?

Marginal likelihood (evidence): $$p(\mathbb{X}|\alpha) = \int_\theta p(\mathbb{X}|\theta) \, p(\theta|\alpha)\ \operatorname{d}\!\theta$$

Prior predictive distribution: $$p(\tilde{x}|\alpha) = \int_{\theta} p(\tilde{x}|\theta) \, p(\theta|\alpha) \operatorname{d}\!\theta$$

• For example, when there are two parameters, one being the nuisance parameter and the other being the parameter of interest, I use the term marginal likelihood for the likelihood integrated over the conditional prior distribution of the nuisance parameter given the parameter of interest. What have you seen exactly ? The likelihood is about a parameter and is not a probability distribution. – Stéphane Laurent Apr 21 '15 at 13:19
• You are talking about frequentist ways of seeing the marginal likelihood not the Bayesian way (i.e, evidence). I have added the definitions to the question. – fsociety Apr 21 '15 at 13:23
• In view of your formulas, you have two parameters $\alpha$ and $\theta$ ? If so, this is exactly what I said in my first comment. – Stéphane Laurent Apr 21 '15 at 14:14
• Yes, you marginalize $\theta$ out. I still do not have an answer to the question though. – fsociety Apr 21 '15 at 14:17
• You are right, my answer has nothing to do with the "marginal likelihood" in the sense of your question. I prefer not to delete it, because it could be useful for someone who wonders about this terminology. – Stéphane Laurent Jul 21 '15 at 18:18

I'm assuming $\alpha$ contains the values that define your prior for $\theta$. When this is the case, we typically omit $\alpha$ from the notation and have the marginal likelihood $$p(\mathbb{X}) = \int p(\mathbb{X}|\theta) p(\theta) d\theta.$$ The prior predictive distribution is not well defined in that you haven't told me what it is that you want predicted, e.g. the prior predictive distribution is different when predicting a single data point and predicting a set of observations. In the notation, this is confusing because $p(\tilde{x}|\theta)$ is different depending on what $\tilde{x}$ is.
For a parametric model ${\cal M} = \{p(\cdot \mid \theta, \alpha)\}$ with two parameters $\theta$ and $\alpha$ equipped with a prior distribution $\pi(\theta, \alpha)$ then the ("joint") likelihood on $(\theta, \alpha)$ after $x$ has been observed is defined by $$L(\theta, \alpha \mid x) \overset{\theta,\alpha}{\propto} p(x \mid \theta, \alpha).$$ See here about my notation $\overset{\theta,\alpha}{\propto}$.
The marginal likelihood on $\alpha$ is obtained by integrating the joint likelihood over the conditional prior distribution $\pi(\theta \mid \alpha)$: $$\tilde L(\alpha \mid x) \overset{\alpha}{\propto} \int L(\theta,\alpha \mid x) \pi(\theta \mid \alpha) d\theta.$$ This is nothing but the "ordinary" likelihood for a new model $\tilde{\cal M} = \{\tilde p(\cdot \mid \alpha)\}$ with parameter $\alpha$, obtained by integrating the original sampling distribution over the conditional prior distribution $\pi(\theta \mid \alpha)$: $$\tilde p(x \mid \alpha) = \int p(x \mid \theta, \alpha)\pi(\theta \mid \alpha) d\theta$$ which is also the conditional prior predictive distribution (of $x$ given $\alpha$). Using the marginal prior distribution $\pi(\alpha)$ of $\alpha$ for this model yields exactly the same posterior distribution: $$\pi(\alpha \mid x) \overset{\alpha}{\propto} \pi(\alpha)\tilde L(\alpha \mid x).$$