How to know if the derivatives exist in Hamiltonian Monte Carlo?

Question

In section 3.2 of Radford Neal's take on HMC he says:

We must also be able to compute the partial derivatives of the log of the density function. These derivatives must therefore exist, except perhaps on a set of points with probability zero, for which some arbitrary value could be returned.

This is w.r.t. the Hamiltonian equations:

\begin{equation} \frac{\partial \theta}{\partial t} = \frac{K (\mathbf{p})}{\partial \mathbf{p}_i} \ \wedge \ \frac{\partial \mathbf{p}_i}{\partial t} = -\frac{U (\theta)}{\partial \theta_i} \end{equation}

where $i \in \{1,\ldots,d\}$, where $d$ are the number dimensions of the parameter array. And $\theta$ are the positions and $\boldsymbol p$ the momenta.

So here is my somewhat embarrassing question: how do we know if these derivatives exist?

Answer 1

The following is a rough exposition of what the requirement for differentiability on the parameters means here.

$U$ involves the log posterior up to an additive constant where $\theta$ are the model parameters. The requirement for differentiability is essentially that one can make a miniscule change to a parameter $\theta$ and this will return a small change in the posterior $U(\theta)$.

Problems of this sort often crop up in some important cases:

• Parameters which can take on only a few discrete values. E.g. counts, categories, binary variables. To see think about what happens if we have $U(\theta)$ defined on $\theta \in \{\ 0, 1\}$. What then happens if we add a small number to our parameter e.g. $\theta' = \theta + 1/100000$? We get a value which is no longer valid to even plug into our $U$.
• Parameters on intervals with at least one boundary, e.g lengths, times etc. (what happens if we take a small step to the left at $\theta=0$ when we have a requirement that $\theta \geq 0$?)

HMC needs this because it operates by adding small changes to $\theta$. The differentiability requirement is, then, roughly that the probability won't fall off a cliff if we make a small change to $\theta$.

Work-arounds exist for many such problems, but will generally require reparameterisation of the likelihood and prior.

EDIT: I focused on the why here because it's not enough to show that a function is not differentiable everywhere (a topic on which CV can offer nothing more than wikipedia, which the OP may wish to look at also). Neal's comment is that the set of discontinuties must have zero probability.

This is hand waving equivalent to saying that one is going to explore a room while blindfolded by taking tentative steps. If the room is booby trapped, but the booby traps are sufficiently sparse, then we'll get lucky and never step on one. If the booby traps are not sufficiently sparse then if we wander around long enough we'll get blown up.

Where being blown up means getting divide by zeros, log of zero, or some other, weirder possibilities.