I'm curious if there are applications of MCMC outside of Bayesian Inference?
this conversation started when discussing Monte Carlo methods vs Markov Chain Monte Carlo methods with a coworker. He asked what the difference was, and the best I could summarize was (A) MCMC is a subset of MC, and (B) MCMC has stricter assumptions about what is or isn't allowed.
I used the example of Casino-style games. Consider a game where a player rolls a die 1-3 times and keeps winnings equal to the final throw. We can price the "fair value" of the game through monte carlo simulations. However, this requires definition of strategy. For example, one simple but effective strategy is to only take a subsequent throw if the previous value is less than the expectation of a given roll (3.5). Once you throw >= 4 you quit. Otherwise, you throw again.
This works well, but makes no assumptions of priors, likelihoods, etc. Conversely, if you want to use an MCMC, the best example that comes to mind is posterior inference. For example, if you're modeling linear regression of weights on heights of men, you supply a prior and likelihood and the sampler is able to return the posterior. In the discrete case, a markov chain can be represented as a matrix of transitions from one state to another. If you let a bot bounce around according to this matrix, it would converge on the "stable distribution" over time. (A simpler method is simply the leading eigenvector of the transition matrix.) Anyway, in the continuous case, we don't have a transition matrix, and so we use a transition kernel to make movement decisions (which generally means comparing the previous/subsequent likelihoods and making a probabilistic decision.)
All that to say, MCMC methods appear to require a transition kernel, which means it requires likelihoods for movements, and this info will converge on the stable distribution given enough movements. By contrast, vanilla MC methods make less assumptions about what the end result should look like. This sounds like a bad working definition and so I'm curious if there are other use cases of MCMC methods? This might help expand my idea of how MCMC and vanilla MC are different.
