This is a bit of a curious situation. I have an energy function $E=S+N$ which is the sum of a smooth differentiable function $S$ and a piecewise constant "noise" function $N$. This means that on average the gradient does point in the right direction, but even a tiny step in the gradient's direction will fall on a different piece.
Is there any way to adapt HMC to handle this situation? For all practical purposes, the "noise" function is a black-box.
So far I'm thinking of using HMC updates based only on the gradient of $S$, with a MH acceptance step at the end. The only difference with regular HMC being that the acceptance ratio may not be 1 (or close to 1). Is this correct or am I missing something?
P.S. I put "noise" in quotes because it does capture the look of the function (low spatial correlation), but it is not noise in the sense that I do not want to ignore its contribution to the energy function.