Regarding Gibbs sampling and HMC in fitting Bayesian model, their differences and advantages I have a question regarding the two MCMC algorithms, Gibbs sampling and Hamiltonian Monte Carlo (HMC) for performing the Bayesian analysis.
If using Gibbs sampling, my understanding is that we need to derive the exact formulas for different conditional distributions corresponding to interested latent variables.  The Gibbs sampling scheme has to rely on these formulas. If we are handling complex models in practice, deriving the exact formula might be infeasible.
If using HMC, we do not need to derive these exact formulas and rely on gradient information, (like probabilistic programming) to fit the model. This makes modeling a complex Bayesian model feasible in practice.  Is this understanding correct?
 A: It is incorrect to state that a Gibbs sampler requires the exact densities of the full conditionals. A Gibbs sampling algorithm requires a collection of conditional distributions that

*

*correspond to a partition of the vector to be simulated (or a completion of said vector by auxiliary variables) into blocks

*and such that the conditional distributions of these blocks are generative models, ie are associated with simulation algorithms of a moderate enough complexity.

In the event the simulation from the conditionals is not readily available, it can be replaced with a Metropolis-within-Gibbs version, which requires further that the joint density associated with these conditionals is available up to a normalising constant. The pseudo-marginal generalisation demonstrates that using an unbiased estimate of the density is also valid.
An HMC algorithm similarly requires the (joint) target density to be available up to a multiplicative  constant so that the gradient be computed (or again estimated in an unbiased manner)]. There is therefore no complexity gain in using HMC.
