Whenever I want to get started with understanding a new statistical topic, I start by reading articles about it. In this case, I'd start with Carpenter et al. "Stan: A Probabilistic Programming Language," in the Journal of Statistical Software which introduces Stan. The first paragraph is enough to get us started.
The goal of the Stan project is to provide a flexible probabilistic programming language for statistical modeling along with a suite of inference tools for fitting models that are robust, scalable, and efficient.
Stan differs from BUGS (Lunn, Thomas, and Spiegelhalter 2000; Lunn, Spiegelhalter, Thomas, and Best 2009; Lunn, Jackson, Best, Thomas, and Spiegelhalter 2012) and JAGS (Plummer 2003) in two primary ways. First, Stan is based on a new imperative probabilistic programming language that is more flexible and expressive than the declarative graphical modeling languages underlying BUGS or JAGS, in ways such as declaring variables with types and supporting local variables and conditional statements. Second, Stan’s Markov chain Monte Carlo (MCMC) techniques are based on Hamiltonian Monte Carlo (HMC), a more efficient and robust sampler than Gibbs sampling or Metropolis Hastings for models with complex posteriors.1
The number at the end is a footnote. It has citations which support the claim made in that sentence. In this case, the footnote reads "Neal (2011) analyzes the scaling benfit of HMC with dimensionality. Hoffman and Gelman (2014) provide practical comparisons of Stan’s adaptive HMC algorithm with Gibbs, Metropolis, and standard HMC samplers," and the citations are
Neal R (2011). “MCMC Using Hamiltonian Dynamics.” In S Brooks, A Gelman, GL Jones, XL Meng (eds.), Handbook of Markov Chain Monte Carlo, pp. 116–162. Chapman and Hall/CRC.
Hoffman MD, Gelman A (2014). “The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo.” Journal of Machine Learning Research, 15(Apr), 1593–1623.
which will elaborate on the differences in more detail.
It's important to point out that the advantages of
stan cited in this passage are not related to the posteriors being different, but are facts about efficiency.
Indeed, one can show that, under certain conditions, Gibbs, MCMC and Metropolis-Hastings will converge to the posterior (albeit it might take far too long for the chains to mix compared to HMC/NUTS), so it would be surprising that HMC/NUTS would differ when these conditions are met.
Bob Carpenter, one of the developers of Stan, provides a concrete example of a case where Stan can solve a problem that Gibbs sampling cannot in this thread on the Stan forums.
[T]here’s an example of how to code exactly this model in the latent discrete parameters chapter of the users guide. You can find this example and others in my latest paper ["Comparing Bayesian Models of Annotation" by
Silviu Paun, Bob Carpenter, Jon Chamberlain, Dirk Hovy, Udo Kruschwitz, Massimo Poesio. Transactions of the Association for Computational Linguistics (2018)], all coded in Stan.
Gibbs is actually a very bad way to fit these models—it’s super slow to converge. These models used to take 24 hours to fit in WinBUGS with very poor mixing and they now fit in like 30 minutes in Stan. Just be careful to use reasonable inits because there’s a non-identifiability. Duco Veen’s visiting us at Columbia from Utrecht and working on a case study that should be out soon.
In other words, if you're trying to estimate this model and you run WinBUGS for 30 minutes, the chains that you get from the WinBUGS model will exhibit poor mixing, the model will not converge, and the samples will not be representative of the posterior density. At that point, you have a choice. You can wait another 23 hours and 30 minutes for the chains to mix, or you can code the model in Stan.
Not all parameterizations, or even all models, are going to be fast to estimate in Stan. There are problematic parameterizations, also discussed in the Stan User Guide, which have a geometry that's very hard for HMC/NUTS to navigate. The User Guide also contains suggested reparameterizations which can ameliorate these problems. This does not imply that all models can be estimated in Stan, or even that Stanwill be more efficient for any particular model; some models are simply challenging, either generically or for Stan specifically.
That said, Stan is a tool that solves some specific problems more quickly compared to popular alternatives. Part of obtaining expertise is knowing how to differentiate among the various alternative tools and methods for solving problems and choosing the tool that is best for the job.