20
$\begingroup$

Does anyone know around what year MCMC became commonplace (i.e., a popular method for Bayesian inference)? A link to the number of published MCMC (journal) articles over time would be especially helpful.

$\endgroup$
  • 2
    $\begingroup$ I doubt that anyone could provide a single year. It's more reasonable to consider the diffusion of MCMC over time. It originated in the 50s with the Metropolis-Hastings algorithm but did not see wide adoption and use until the advent of relatively inexpensive computational power beginning in the 80s. To the best of my knowledge the first uses were in Bayesian facial recognition technologies of that time. Secondarily, beginning in the 90s, MCMC use spread to other fields such as economics & marketing with the Chicago school. Check out Gilks & Spiegelhalter's 1996 Practical MCMC. $\endgroup$ – user332577 Oct 7 '19 at 18:06
  • 1
    $\begingroup$ This question is vague and calls for opinion (there's no accepted definition of commonplace or popular). It admits any number of arguably correct answers. $\endgroup$ – Glen_b Oct 7 '19 at 23:11
  • 1
    $\begingroup$ @Glen_b I think the answer given below is excellent. Do you disagree? Or did you write your comment before that answer? (Both just say 'yesterday'). $\endgroup$ – Peter Flom Oct 9 '19 at 10:53
  • 2
    $\begingroup$ @Peter Mine came before either answer; hover your mouse over the word "yesterday" on each (or anything indicating an elapsed time since posting) to see the precise UTC time. I think the answer you indicate is a good partial answer but the question would still admit several completely different takes with no good basis to choose between them. $\endgroup$ – Glen_b Oct 9 '19 at 12:26
26
$\begingroup$

This paper by Christian (Xi'an) Robert and George Casella provides a nice summary of the history of MCMC. From the paper (emphasis is mine).


What can be reasonably seen as the first MCMC algorithm is what we now call the Metropolis algorithm, published by Metropolis et al. (1953). It emanates from the same group of scientists who produced the Monte Carlo method, namely, the research scientists of Los Alamos, mostly physicists working on mathematical physics and the atomic bomb.


The Metropolis algorithm was later generalized by Hastings (1970) and his student Peskun (1973,1981)


Although somewhat removed from statistical inference in the classical sense and based on earlier techniques used in Statistical Physics, the landmark paper by Geman and Geman (1984) brought Gibbs sampling into the arena of statistical application. This paper is also responsible for the name Gibbs sampling


In particular, Geman and Geman (1984) influenced Gelfand and Smith (1990) to write a paper that is the genuine starting point for an intensive use of MCMC methods by the main-stream statistical community. It sparked new inter-est in Bayesian methods, statistical computing, algorithms and stochastic processes through the use of computing algorithms such as the Gibbs sampler and the Metropolis–Hastings algorithm.


Interestingly, the earlier paper by Tanner and Wong (1987) had essentially the same ingredients as Gelfand and Smith (1990), namely, the fact that simulating from the conditional distributions is sufficient to asymptotically simulate from the joint.This paper was considered important enough to be a discussion paper in the Journal of the American Statistical Association, but its impact was somehow limited, compared with Gelfand and Smith (1990).


I couldn't find the number of journal articles published over time, but here is a Google Ngram plot for the number of mentions over time. It more or less agrees with the notion that MCMC became commonplace after the 1990 paper of Gelfand and Smith.

enter image description here

Reference

Robert, Christian, and George Casella. "A short history of Markov chain Monte Carlo: Subjective recollections from incomplete data." Statistical Science (2011): 102-115.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Thank you! I would deem 1990 to be the most important date in the history of MCMC, as four papers by Alan Gelfand and Adrian Smith appeared that very year in top Statistics journals and made the concept of using Markov chains for simulation suddenly mainstream. I remember attending a talk by Adrian Smith in June 1989 in Seherbrooke (PQ) where he demonstrated the universality of the idea by showing a slide with a few lines of (Fortran?) code. $\endgroup$ – Xi'an Oct 9 '19 at 20:41
12
$\begingroup$

The excellent answer by knrumsey gives some history on the progression of important academic work in MCMC. One other aspect worth examining is the development of software to facilitate MCMC by the ordinary user. Statistical methods are often used mostly by specialists until they are implemented in software that allows the ordinary user to implement them without programming. For example, the software BUGS had its first release in 1997. That does not appear to have changed the growth trajectory in the N-Grams plot, but it may have been an influence in bringing the method into common usage among those users who found it intimidating to program their own routines.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Huh, there is a small twist in the line for MCMC right around 1997. $\endgroup$ – muru Oct 9 '19 at 2:05
  • $\begingroup$ Well-spotted - not sure if it would be a big enough change to be statistically significant, but noted anyway. $\endgroup$ – Ben Oct 9 '19 at 2:07
  • $\begingroup$ Visually estimating, if the slope before 1997 were maintained, we would have seen something like 0.000015 % around 2004 (but the actual value is close to 0.0000225 %). That's a 50% increase. But I suppose the numbers are too small anyway. $\endgroup$ – muru Oct 9 '19 at 2:13
  • $\begingroup$ Perhaps you're right - good eyes! $\endgroup$ – Ben Oct 9 '19 at 2:42
  • 1
    $\begingroup$ hmmm, BUGS was presented at the Valencia Bayesian Statistics conference in 1991. $\endgroup$ – Xi'an Oct 9 '19 at 20:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.