In the most basic sense, what is marginal likelihood? I understand that likelihood differs from a probability distribution because likelihood describes the probability of certain parameter values given the data that you've observed (it's essentially a distribution that describes observed data) while a probability distribution describes the probability of observing certain values given constant parameter values. But what is a marginal likelihood and how does it relate to posterior distributions? (preferably explained without, or with as little as possible, probability notation so that the explanation is more intuitive). Any examples would be great as well.
 A: In Bayesian statistics, the marginal likelihood
$$m(x) = \int_\Theta f(x|\theta)\pi(\theta)\,\text d\theta$$
where

*

*$x$ is the sample

*$f(x|\theta)$ is the sampling density, which is proportional to the model likelihood

*$\pi(\theta)$ is the prior density

is a misnomer in that

*

*it is not a likelihood function [as a function of the parameter], since the parameter is integrated out (i.e., the likelihood function is averaged against the prior measure),

*it is a density in the observations, the predictive density of the sample,

*it is not defined up to a multiplicative constant,

*it does not solely depend on sufficient statistics

Other names for $m(x)$ are evidence, prior predictive, partition function. It has however several important roles:

*

*this is the normalising constant of the posterior distribution$$\pi(\theta|x) = \dfrac{f(x|\theta)\pi(\theta)}{m(x)}$$

*in model comparison, this is the contribution of the data to the posterior probability of the associated model and the numerator or denominator in the Bayes factor.

*it is a measure of goodness-of-fit (of a model to the data $x$), in that $2\log m(x)$ is asymptotically the BIC (Bayesian information criterion) of Schwarz (1978).

See also
Normalizing constant in Bayes theorem
Normalizing constant irrelevant in Bayes theorem?
Intuition of Bayesian normalizing constant
A: Although the OP clearly refers to the Bayesian framework, it may be
worth mentioning that the expression marginal likelihood has been
used for long outside of the Bayesian framework, with a different
meaning. This frequentist concept is described in several famous
books, especially in those by D.R. Cox and coauthors. A modern
presentation is to be found in the book Statistical Models by
A.C. Davison (Chap. 12) which inspired this answer. The book
Principles of Statistical Inference by D.R. Cox discusses both the
frequentist and the Bayesian concepts.
Consider a random vector $\mathbf{Y}$ of observations with
distribution depending on a vector of parameters
$\boldsymbol{\theta}$. If $\mathbf{Y}$ splits into two sub-vectors,
say $\mathbf{V}$ and $\mathbf{W}$, the likelihood is
$$
L(\boldsymbol{\theta};\,\mathbf{y}) = 
f_{\mathbf{Y}}(\mathbf{y};\,\boldsymbol{\theta}) = 
f_{\mathbf{V}}(\mathbf{v}; \boldsymbol{\theta}) \,
    f_{\mathbf{W} \vert \mathbf{V}}(\mathbf{w} \, \vert\, \mathbf{v}; \, 
 \boldsymbol{\theta}). \tag{1}
$$
If instead the couple $[\mathbf{V}, \mathbf{W}]$ is a sufficient
statistic for $\boldsymbol{\theta}$, the same form holds, up to a
multiplicative constant (w.r.t. $\boldsymbol{\theta}$). It may happen
that the information on some specific component of interest in
$\boldsymbol{\theta}$ is mainly conveyed by one of the two factors of
the product on which we can therefore focus.
As an example consider
$\mathbf{Y} \sim_{\text{i.i.d.}}  \text{Norm}(\mu,\, \sigma^2)$, so that
$\boldsymbol{\theta} := [\mu,\,\sigma^2]$, and
assume that we are interested in the inference on $\sigma^2$ only.
It can be checked that
$$
    f_{\mathbf{Y}}(\mathbf{y}; \, \sigma^2,\,\mu) = 
 f_{S^2} (s^2; \, \sigma^2) \,
 f_{\bar{Y}} (\bar{y}; \, \sigma^2,\, \mu) \,
 f_{\mathbf{Y}\vert \bar{Y},S^2}(\mathbf{y} \, \vert \, \bar{y}, \, s^2).
$$
At the right-hand side the third term does not depend on the
parameter: in other words, the
sample mean $\bar{Y}$ and the sample variance $S^2$ are jointly
sufficient statistics for $\boldsymbol{\theta}$.
It can be anticipated that the information on
$\sigma^2$ in the likelihood is conveyed by the first term since the sample mean
$\bar{Y}$ does not tell much about $\sigma^2$. So the inference on $\sigma^2$ can be based on $f_{S^2} (s^2;
\, \sigma^2)$ called the/a marginal likelihood for this specific case.
More generally, the form (1) can have a special importance when the
parameter vector $\boldsymbol{\theta} = [\boldsymbol{\psi},\,
\boldsymbol{\lambda}]$ splits into two parts: a parameter of interest
$\boldsymbol{\psi}$ and a nuisance parameter
$\boldsymbol{\lambda}$. We can call marginal likelihood a function
which only depends on $\boldsymbol{\psi}$ and on the observations
available, and which extracts most of the information on
$\boldsymbol{\psi}$ that can be retrieved from the observations. If
the joint density can be factored as
$$
   f_\mathbf{Y}(\mathbf{y}; \boldsymbol{\psi},\, \boldsymbol{\lambda}) 
   = f_{\mathbf{V}}(\mathbf{v}; \boldsymbol{\psi}) \,
    f_{\mathbf{W} \vert \mathbf{V}}(\mathbf{w} \, \vert\, \mathbf{v}; \, 
 \boldsymbol{\psi}, \,\boldsymbol{\lambda}), \tag{M}
$$
then we can ignore the second term at the r.h.s. and use the following
marginal likelihood (for $\boldsymbol{\psi}$)
$L_{\text{M}}(\boldsymbol{\psi};\, \mathbf{y}) :=
f_{\mathbf{V}}(\mathbf{v}; \boldsymbol{\psi})$.
A different form of factorisation that can be used is
$$
   f_\mathbf{Y}(\mathbf{y}; \boldsymbol{\psi},\, \boldsymbol{\lambda}) 
   = f_{\mathbf{V}}(\mathbf{v}; \boldsymbol{\psi},\, \boldsymbol{\lambda}) \,
    f_{\mathbf{W} \vert \mathbf{V}}(\mathbf{w} \, \vert \, \mathbf{v}; \, \boldsymbol{\psi}).
 \tag{C}
$$
In this case we may base the inference on $\boldsymbol{\psi}$ on the
so-called the conditional likelihood
$L_{\text{C}}(\boldsymbol{\psi};\, \mathbf{y}) := f_{\mathbf{W} \vert
\mathbf{V}}(\mathbf{w} \, \vert \, \mathbf{v}; \,
\boldsymbol{\psi})$. So, while in (M) the parameter of interest is
isolated in the marginal part of the factorisation, in (C) it is
isolated in the conditional part. As a typical example is provided by
regression where the inference is usually conditional on the vector of
covariates.
So in its usual frequentist meaning, a marginal likelihood is a
function of the parameter of interest $\psi$ that can be used as a
likelihood to infer on $\psi$ disregarding the nuisance parameter.  A
striking point is that in some cases the inference based on the
marginal likelihood is better than that based on the profile
likelihood, which is mainly asymptotic. This is the case in the
example above, which generalises to linear regression.
A: In my mind the most intuitive role of marginal likelihood is indeed as a normalization factor. I'll elaborate more on this.
The bayes theorem is: $$ P(\phi|X) = \frac{P(X|\phi)P(\phi)} {P(X)} $$
Now let's explore the numerator components:
$$P(X|\phi)P(\phi)$$ This is the prior, multiply by the likelihood using a specific parameter - How well can we explain the data using this specific $\phi$ parameter.
$$P(X)=\int_{\theta}P(X|\theta)P(\theta)\mathrm{d}\theta$$ The marginal likelihood - How well we can explain the data using all the parameters, weighted by the prior.
The ratio between them should give the proportional share of this specific parameter $\phi$. This makes the numerator over the different parameters sum to 1 constructing a probability distribution. Thus the posterior, the ratio between the numerator & denominator, represents the probability distribution over the parameters. 
If for example a specific parameter $\phi$ is very good at explaining the data compared to the other weights, then this parameter will have higher probability.
