# Tag Info

Accepted

### Hamiltonian Monte Carlo vs. Sequential Monte Carlo

Hamiltonian Monte Carlo performs well with continuous target distributions with "weird" shapes. It requires the target distribution to be differentiable as it basically uses the slope of the target ...
• 494

### What is the purpose of "transformed variables" in Stan?

Objects declared in the transformed parameters block of a Stan program are: Unknown but are known given the values of the objects in the ...
• 2,008
Accepted

### For Hamiltonian Monte Carlo, why does negating the momentum variables result in a symmetric proposal?

One of the reasons why the original construction of Hamiltonian Monte Carlo can be tricky to understand is that it is more restrictive than necessary, if only to simplify the theoretical proofs. In ...
Accepted

### Hamiltonian monte carlo

I believe the most up-to-date source on Hamiltonian Monte Carlo, its practical applications and comparison to other MCMC methods is this 2017-dated review paper by Betancourt: The ultimate ...
• 8,269
Accepted

### Hamiltonian Monte Carlo: how to make sense of the Metropolis-Hasting proposal?

The deterministic Hamiltonian trajectories are useful only because they are consistent with the target distribution. In particular, trajectories with a typical energy project onto regions of high ...
Accepted

### Understanding the Typical Set for Markov chain Monte Carlo sampling

$\mathrm{d}q$ is uniform across the entire space and that's the problem! Unfortunately as we consider higher-dimensional spaces out intuition of uniform starts failing us and we end up in conceptual ...
Accepted

### Hamiltonian Monte Carlo (HMC): what's the intuition and justification behind a Gaussian-distributed momentum variable?

It's not so much that we are after $\pi(E)$, it's just that if $\pi(E)$ and $\pi(E|q)$ are dissimilar then our exploration will be limited by our inability to explore all of the relevant energies. ...
Accepted

### Plotting the typical set of a Gaussian distribution

One of the confusing things about concentration of measure is that we're trying to demonstrate deviations away from our naive, low-dimensional intuition. Here that is demonstrated in how the radial ...
Accepted

### How does Hamiltonian Monte Carlo work?

Before answering the question about an intuitive way to think about Hamiltonian Monte Carlo, it's probably best to get a really firm grasp on regular MCMC. Let's set aside the satellite metaphor for ...
• 1,123
Accepted

• 1,346
Accepted

### What could lead to this misbehavior for the expected sample size (ESS)?

This is actually not an error - it is possible for the effective sample size to be larger than the actual sample size. This means that your MCMC samples provide more information about the parameter, ...
• 131

### Proposal distribution in Hamiltonian Monte Carlo

The proposal distribution in Hamiltonian Monte Carlo does not have an explicit form in general. Instead, samples from it are defined operationally: first sample an initial velocity and then move the ...
• 165
Accepted

### Is there an HMC algorithm that estimates a model with noncontinuous parameters?

Currently, no- such a solution does not exist. Core developers on PyMC3 actually addressed this, noting that it's a high impact problem but the solution remains over the horizon. (I'll dig for a ...
• 2,945
1 vote

This is only a partial answer, but in general you can go a long way by adding an order constraint to your model: enforcing $\theta_1 \lt \theta_2 \lt\dots\lt \theta_k$. This is trickier to do if $\... • 9,087 1 vote ### volume preservation in MCMC Better late than never. I also thought about this question when I first came across the paper, and my self-given answer is the following. In MCMC, if we are at$x$and we get at$y$what we want to ... 1 vote Accepted ### Regarding Gibbs sampling and HMC in fitting Bayesian model, their differences and advantages It is incorrect to state that a Gibbs sampler requires the exact densities of the full conditionals. A Gibbs sampling algorithm requires a collection of conditional distributions that correspond to a ... • 106k 1 vote ### Can someone explain how dual averaging helps the No U-Turn Sampler (NUTS) choose step-size adaptively? I stumbled on this while looking for an answer to my own misunderstanding of the step size algorithm, so your mileage my vary on this answer... With that said, the idea is that the slice sampler is ... • 21 1 vote ### Can I do HMC with the wrong Hamiltonian? If I understood well, the critical point in the second step of the HMC algorithm is that the proposal is volume preserving and reversible, but I am free to use another position energy function than ... • 1,346 1 vote ### Strange substitution in HMC If$K$is a Markov kernel(with density$k$) with stationary distribution$P$(with density$p$), then, if$(X_t)_t$is a stationary Markov chain associated with$K, \begin{align*}\mathbb E\left[\frac{... • 106k 1 vote Accepted ### NUTS algorithm efficient transition kernel The way to interpret this is as a function that could be applied to any potentialw'$. Specifically, When$\lvert \mathcal C^\mathit{new} \rvert > \lvert \mathcal C^\mathit{old} \rvert$,$T$... • 24.8k 1 vote ### For Hamiltonian Monte Carlo, why does negating the momentum variables result in a symmetric proposal? Below is only my opinion since I am also a beginner to HMC. I think it's already clear in your desciption. The negation of momentum is unnecessary in practice because we actually only want the ... 1 vote Accepted ### How to know if the derivatives exist in Hamiltonian Monte Carlo? The following is a rough exposition of what the requirement for differentiability on the parameters means here.$U$involves the log posterior up to an additive constant where$\theta\$ are the model ...
• 4,226

Only top scored, non community-wiki answers of a minimum length are eligible