# Nested sampling: Bayesian evidence integral transformation proof

I am trying to work out how the Bayesian evidence integral ($\equiv Z$) is transformed to a one-dimensional integral as presented in John Skilling's nested sampling paper (page 2): https://projecteuclid.org/download/pdf_1/euclid.ba/1340370944

I'll quote his steps here:

We want to evaluate $$\label{one} \tag{1} Z = \int_{\hat{y}} L(y) \pi(y) \mathrm{d}y,$$ where $y$ is a parameter (possibly a vector of parameters) $L(y)$ is the likelihood function and $\pi(y)$ is the prior function. $\hat{y}$ is the support of $\pi(y)$.

Skilling then defines an infinitesimal unit of `prior mass' $\mathrm{d}X$ to be $$\label{two}\tag{2} \mathrm{d}X = \pi(y) \mathrm{d}y$$ and $$\label{three}\tag{3} X(\lambda) = \int_{L(y) > \lambda} \pi(y) \mathrm{d}y.$$ i.e. $X(\lambda)$ is the integral over the prior at all points for which $L(y) > \lambda$ is satisfied, which means $X \in [0, 1]$.

The method makes sense up to this point, but then he sets $L(X)$ equal to the inverse of equation \ref{three} ($L(X) \equiv \lambda(X)$) and from that deduces

$$\label{four}\tag{4} Z = \int_{0}^{1} L(X) \mathrm{d}X.$$

Intuitively I can see how equation \ref{four} arises, but I cannot see how it is mathematically derived from inverting equation \ref{three}. Could anyone provide a proof for this relation?

I have tried interpreting the integral on the RHS of equation \ref{three} as $P(y | L(Y) > \lambda)$ where $P(\cdot|\cdot)$ is the conditional CDF of the prior, and applying the inverse function theorem $(L'(X) = 1 / \pi(y | L(Y) > \lambda)$ but I feel like I am going nowhere fast.

Is it possible to show this mathematically without a knowledge of measure theory? If not, could you describe the relevant theory to look up to understand the problem more.

Thanks

If $L(y) = \lambda$ for more than one value of $y$ (which is generally the case for likelihood functions), can you still swap the order of integration between $y$ and $\lambda$?

My guess is you can swap the order, since if you split up the integration over $y$ into $n$ separate regions where in each region you get one occurrence of $L(y) = \lambda$, then: $$\label{five}\tag{5} \int_{\hat y} \pi(y) \delta(L(y) - \lambda) \mathrm{d}y = \int_{\hat y_1} \pi(y) \delta(L(y) - \lambda) \mathrm{d}y~+~ ...~+~\int_{\hat y_n} \pi(y) \delta(L(y) - \lambda) \mathrm{d}y,$$ then the double integral becomes $$\label{six}\tag{6} -\int \lambda \int_{\hat y} \pi(y) \delta(L(y) - \lambda) \mathrm{d}y \mathrm{d}\lambda = -\int \lambda \left[ \int_{\hat y_1} \pi(y) \delta(L(y) - \lambda) \mathrm{d}y \quad+ ~... ~+ \int_{\hat y_n} \pi(y) \delta(L(y) - \lambda) \mathrm{d}y \right] \mathrm{d} \lambda,$$ and the order of each integral can be re-arranged (I think) $$\label{seven}\tag{7} -\int_{\hat y_1} \pi(y) \left[ \int \lambda \delta(L(y) - \lambda) \mathrm{d}\lambda \right] \mathrm{d}y~+ ~...~+~\int_{\hat y_n} \pi(y) \left[ \int \lambda \delta(L(y) - \lambda) \mathrm{d} \lambda \right] \mathrm{d}y,$$ which would also lead to the desired result.

I am trying to relate this to Fubini's theorem, but I am unsure how it is affected by the fact that we are integrating over a delta function.

I guess a more general question I have following Bridgeburner's answer is, how does integrating over a delta function affect the applicability of Fubini's theorem?

The expression for $X$ can be rewritten:

$$X(\lambda) = \int_\hat{y} \pi(y) \Theta(L(y) - \lambda) \mathrm{d}y,$$

where $\Theta(x)$ is the Heaviside step function, which satisfies $\frac{\mathrm{d}}{\mathrm{d}x} \Theta(x) = \delta(x),$ the Dirac delta function. So,

$$\frac{\mathrm{d}X}{\mathrm{d}\lambda} = -\int_\hat{y} \pi(y) \delta(L(y) - \lambda) \mathrm{d}y.$$

Now the final form of the integral, as you expressed it, can be re-expressed as an integral over $\lambda$ by making a change of variables from $X$ to $\lambda$, $$\begin{split} Z &= \int_0^1 \lambda(X) \mathrm{d}X \\ &= \int \lambda \cdot \left( \frac{\mathrm{d}X}{\mathrm{d}\lambda} \right) \mathrm{d}\lambda \\ &= -\int \lambda \int_{\hat y} \pi(y) \delta(L(y) - \lambda) \mathrm{d}y \mathrm{d}\lambda. \end{split}$$ Because the outer integral is over all possible values of $\lambda,$ and the inner integral is over all $\hat y,$ there is no dependence of the bounds on each others' variable, so the order of integration can simply be switched.

$$\begin{split} Z &= \int_{\hat y} \pi(y) \left( -\int \lambda \delta(L(y) - \lambda) \mathrm{d} \lambda \right) \mathrm{d}y \\ &= \int_{\hat y} \pi(y) L(y) \mathrm{d}y. \end{split}$$

• Hi thank you for your answer, I have a question with regards to you swapping the order of integration but my working is too long to include here so I will put it at the end of my original question. – KamKam Mar 16 '18 at 23:47