# How to know if the derivatives exist in Hamiltonian Monte Carlo?

We must also be able to compute the partial derivatives of the log of the density function. These derivatives must therefore exist, except perhaps on a set of points with probability zero, for which some arbitrary value could be returned.

This is w.r.t. the Hamiltonian equations:

$$\frac{\partial \theta}{\partial t} = \frac{K (\mathbf{p})}{\partial \mathbf{p}_i} \ \wedge \ \frac{\partial \mathbf{p}_i}{\partial t} = -\frac{U (\theta)}{\partial \theta_i}$$

where $i \in \{1,\ldots,d\}$, where $d$ are the number dimensions of the parameter array. And $\theta$ are the positions and $\boldsymbol p$ the momenta.

So here is my somewhat embarrassing question: how do we know if these derivatives exist?

• You can examine the expression for the density to see whether it's differentiable Commented Mar 6, 2017 at 18:14

The following is a rough exposition of what the requirement for differentiability on the parameters means here.

$U$ involves the log posterior up to an additive constant where $\theta$ are the model parameters. The requirement for differentiability is essentially that one can make a miniscule change to a parameter $\theta$ and this will return a small change in the posterior $U(\theta)$.

Problems of this sort often crop up in some important cases:

• Parameters which can take on only a few discrete values. E.g. counts, categories, binary variables. To see think about what happens if we have $U(\theta)$ defined on $\theta \in \{\ 0, 1\}$. What then happens if we add a small number to our parameter e.g. $\theta' = \theta + 1/100000$? We get a value which is no longer valid to even plug into our $U$.
• Parameters on intervals with at least one boundary, e.g lengths, times etc. (what happens if we take a small step to the left at $\theta=0$ when we have a requirement that $\theta \geq 0$?)

HMC needs this because it operates by adding small changes to $\theta$. The differentiability requirement is, then, roughly that the probability won't fall off a cliff if we make a small change to $\theta$.

Work-arounds exist for many such problems, but will generally require reparameterisation of the likelihood and prior.

EDIT: I focused on the why here because it's not enough to show that a function is not differentiable everywhere (a topic on which CV can offer nothing more than wikipedia, which the OP may wish to look at also). Neal's comment is that the set of discontinuties must have zero probability.

This is hand waving equivalent to saying that one is going to explore a room while blindfolded by taking tentative steps. If the room is booby trapped, but the booby traps are sufficiently sparse, then we'll get lucky and never step on one. If the booby traps are not sufficiently sparse then if we wander around long enough we'll get blown up.

Where being blown up means getting divide by zeros, log of zero, or some other, weirder possibilities.

• So if I am only operating with Gaussians, which are continuous, they always have derivatives, and HMC will thus not have any problems? Commented Mar 6, 2017 at 21:54
• For example, if your prior is normal with known variance, you're fine. If your prior involves an unknown variance, you may encounter problem II because variances must be non-negative. This is often worked around by using log-scale prior on the variance. Commented Mar 6, 2017 at 23:03
• Ahh I see, so if I were to a multivariate Gaussian with a NIW prior on the covariance matrix, I would basically be in trouble? Commented Mar 7, 2017 at 11:30
• With HMC you'd probably need to take corrective action to handle the wishart params. NIW prior is very tractable so there are other solutions. E.g. it is conjugate, so unless there's some funky additions, you don't need to sample the posterior mean and variance there's an analytical form to them. Commented Mar 7, 2017 at 14:31
• That makes sense. So trouble only happens, which I think is what you were trying to explain, when there is no analytical form for the updates. Commented Mar 7, 2017 at 15:03