0
$\begingroup$

I am trying to work out how the Bayesian evidence integral ($\equiv Z$) is transformed to a one-dimensional integral as presented in John Skilling's nested sampling paper (page 2): https://projecteuclid.org/download/pdf_1/euclid.ba/1340370944

I'll quote his steps here:

We want to evaluate \begin{equation} \label{one} \tag{1} Z = \int_{\hat{y}} L(y) \pi(y) \mathrm{d}y, \end{equation} where $y$ is a parameter (possibly a vector of parameters) $L(y)$ is the likelihood function and $\pi(y)$ is the prior function. $\hat{y}$ is the support of $\pi(y)$.

Skilling then defines an infinitesimal unit of `prior mass' $\mathrm{d}X$ to be \begin{equation} \label{two}\tag{2} \mathrm{d}X = \pi(y) \mathrm{d}y \end{equation} and \begin{equation} \label{three}\tag{3} X(\lambda) = \int_{L(y) > \lambda} \pi(y) \mathrm{d}y. \end{equation} i.e. $X(\lambda) $ is the integral over the prior at all points for which $L(y) > \lambda$ is satisfied, which means $X \in [0, 1]$.

The method makes sense up to this point, but then he sets $L(X)$ equal to the inverse of equation \ref{three} ($L(X) \equiv \lambda(X)$) and from that deduces

\begin{equation} \label{four}\tag{4} Z = \int_{0}^{1} L(X) \mathrm{d}X. \end{equation}

Intuitively I can see how equation \ref{four} arises, but I cannot see how it is mathematically derived from inverting equation \ref{three}. Could anyone provide a proof for this relation?

I have tried interpreting the integral on the RHS of equation \ref{three} as $P(y | L(Y) > \lambda)$ where $P(\cdot|\cdot)$ is the conditional CDF of the prior, and applying the inverse function theorem $(L'(X) = 1 / \pi(y | L(Y) > \lambda)$ but I feel like I am going nowhere fast.

Is it possible to show this mathematically without a knowledge of measure theory? If not, could you describe the relevant theory to look up to understand the problem more.

Thanks

EDIT: Response to Bridgeburner's answer

If $L(y) = \lambda$ for more than one value of $y$ (which is generally the case for likelihood functions), can you still swap the order of integration between $y$ and $\lambda$?

My guess is you can swap the order, since if you split up the integration over $y$ into $n$ separate regions where in each region you get one occurrence of $L(y) = \lambda$, then: \begin{equation} \label{five}\tag{5} \int_{\hat y} \pi(y) \delta(L(y) - \lambda) \mathrm{d}y = \int_{\hat y_1} \pi(y) \delta(L(y) - \lambda) \mathrm{d}y~+~ ...~+~\int_{\hat y_n} \pi(y) \delta(L(y) - \lambda) \mathrm{d}y, \end{equation} then the double integral becomes \begin{equation} \label{six}\tag{6} -\int \lambda \int_{\hat y} \pi(y) \delta(L(y) - \lambda) \mathrm{d}y \mathrm{d}\lambda = -\int \lambda \left[ \int_{\hat y_1} \pi(y) \delta(L(y) - \lambda) \mathrm{d}y \quad+ ~... ~+ \int_{\hat y_n} \pi(y) \delta(L(y) - \lambda) \mathrm{d}y \right] \mathrm{d} \lambda, \end{equation} and the order of each integral can be re-arranged (I think) \begin{equation} \label{seven}\tag{7} -\int_{\hat y_1} \pi(y) \left[ \int \lambda \delta(L(y) - \lambda) \mathrm{d}\lambda \right] \mathrm{d}y~+ ~...~+~\int_{\hat y_n} \pi(y) \left[ \int \lambda \delta(L(y) - \lambda) \mathrm{d} \lambda \right] \mathrm{d}y, \end{equation} which would also lead to the desired result.

I am trying to relate this to Fubini's theorem, but I am unsure how it is affected by the fact that we are integrating over a delta function.

I guess a more general question I have following Bridgeburner's answer is, how does integrating over a delta function affect the applicability of Fubini's theorem?

$\endgroup$
1
$\begingroup$

The expression for $X$ can be rewritten:

$$ X(\lambda) = \int_\hat{y} \pi(y) \Theta(L(y) - \lambda) \mathrm{d}y, $$

where $\Theta(x)$ is the Heaviside step function, which satisfies $\frac{\mathrm{d}}{\mathrm{d}x} \Theta(x) = \delta(x),$ the Dirac delta function. So,

$$ \frac{\mathrm{d}X}{\mathrm{d}\lambda} = -\int_\hat{y} \pi(y) \delta(L(y) - \lambda) \mathrm{d}y. $$

Now the final form of the integral, as you expressed it, can be re-expressed as an integral over $\lambda$ by making a change of variables from $X$ to $\lambda$, $$ \begin{split} Z &= \int_0^1 \lambda(X) \mathrm{d}X \\ &= \int \lambda \cdot \left( \frac{\mathrm{d}X}{\mathrm{d}\lambda} \right) \mathrm{d}\lambda \\ &= -\int \lambda \int_{\hat y} \pi(y) \delta(L(y) - \lambda) \mathrm{d}y \mathrm{d}\lambda. \end{split} $$ Because the outer integral is over all possible values of $\lambda,$ and the inner integral is over all $\hat y,$ there is no dependence of the bounds on each others' variable, so the order of integration can simply be switched.

$$ \begin{split} Z &= \int_{\hat y} \pi(y) \left( -\int \lambda \delta(L(y) - \lambda) \mathrm{d} \lambda \right) \mathrm{d}y \\ &= \int_{\hat y} \pi(y) L(y) \mathrm{d}y. \end{split} $$

$\endgroup$
  • $\begingroup$ Hi thank you for your answer, I have a question with regards to you swapping the order of integration but my working is too long to include here so I will put it at the end of my original question. $\endgroup$ – KamKam Mar 16 '18 at 23:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.