Intuitively, what does the warning "There were 214 divergent transitions after warmup." mean?
I understand that the samples obtained are useless, and that increasing adapt_delta, and max_treedepth, and lowering stepsize can help. In addition, reparameterizing can also help.
I would like to know what is actually happening when a divergent transition occurs.
In Section 14.5 Divergent Transitions of the Stan Reference Manual it states "The positions along the simulated trajectory after the Hamiltonian diverges will never be selected as the next draw of the MCMC algorithm". Does that mean that after a divergent transition, the subsequent transitions are all rejected? Hence the last accepted proposal (and the current state) will be returned many times in the posterior sample?
I have noticed that I am getting many divergent transitions when working with hierarchical models (this is well documented in blogs on the internet), but some of my parameters are (well seem to be) converging and others are not. But, in a joint model, each proposition should be a parameter vector, and if this proposal is rejected after a divergent transition (and all others thereafter as described above), shouldn't all of the marginal distributions be divergent?
Diagnostics plots seem to show that, say $\mu$ has converged and $\sigma$ has not. One example I had was that $\sigma$ had zero standard deviation. I assumed that this meant that there was a divergent transition after the first proposal and hence all other proposals for $\sigma$ were rejected and consequently, $\sigma$ only had one unique value in its returned sample. But, my sample for $\mu$ was (or seemed) fine. But shouldn't the proposal $(\mu, \sigma)$ have been rejected after the first proposal and hence the $\mu$ sample should also only have one unique value.
Now consider a model with just one parameter. Imagine that my initial proposition is far from the region(s) of high posterior density, and suppose that this chain never reaches the regions with high posterior density. Is this when Stan returns the message saying "There were x divergent transitions after warmup."? Since it can see that the MCMC code has not sampled "enough" from the posterior distribution of interest?
One reason that I think this intuition is wrong is because I have ran MCMC code in the past which has not converged (simply due to not being ran for enough iterations) and I have not received this error message.
Can anyone give an intuitive interpretation of what is going on when a transition diverges? What happens if the proposal eventually reaches the region of high posterior density? Is it possible to find where the MCMC sampler diverged and then discard the samples up to when they are convergent again?
If the Hamiltonian diverges the positions along the simulated trajectory will never be accepted. If I have 214 divergent transitions, this must mean that I entered a phase of divergence and then switched back to a convergence phase and then to another phase of divergence. Otherwise, I should only have one divergent transition. This is difficult to understand since according to the manual, no proposals are accepted after a trajectory diverges and so after a trajectory diverges, shouldn't it stay divergent until the end? Also, why does the MCMC code not just terminate after a divergent trajectory is found? How does x divergent transitions accumulate?