I know that Monte Carlo is used to approximate an integral by sampling. I also learnt MCMC algorithms such as Metropolis-Hastings and Gibbs sampling but I don't know where the "Monte Carlo" part is in those algorithms. Those algorithms are just sampling.
$\begingroup$
$\endgroup$
3
-
4$\begingroup$ en.wikipedia.org/wiki/Monte_Carlo_method Monte Carlo methods are just sampling, usually with the goal of using certain facts about how the sampling was conducted to draw inferences about a problem that would be challenging to solve directly. $\endgroup$– Sycorax ♦Commented Sep 25, 2019 at 12:57
-
$\begingroup$ @Sycorax Thank you for the answer. There is only one thing that is confusing to me. Does "Monte Carlo" concern itself with "APPROXIMATING an integral (given that we know how to sample from the distribution)" or "SAMPLING from an unknown weird distribution"? I am asking because I saw some sources on the internet that say that Monte Carlo is an approximation technique. $\endgroup$– floydCommented Sep 25, 2019 at 17:52
-
1$\begingroup$ Monte Carlo methods include both. On the page that I linked, the experiment about approximating $\pi$ can also be viewed as a MC method to approximate an integral (the area of the circular section). And one application of MC for Bayesian statistics is sampling from an inconvenient target distribution. $\endgroup$– Sycorax ♦Commented Sep 25, 2019 at 18:53
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
As I explained earlier this week in my introductory lecture to Monte Carlo methods, the founding principle of such numerical methods is the Law of Large Numbers, or the stabilisation of empirical frequencies to their expectations. Markov chain Monte Carlo algorithms are a special case of methods implementing the Monte Carlo principle, in that Markov chains are created with the specific purpose of preserving the convergence of empirical averages, which is then called the ergodic theorem. And with the rationale that these Markov chains are easier to produce than iid simulations from the given target.
Further links on this forum on the meaning of Monte Carlo:
-
$\begingroup$ Thank you for the answer. There is only one thing that is confusing to me. Does "Monte Carlo" concern itself with "APPROXIMATING an integral (given that we know how to sample from the distribution)" or "SAMPLING from an unknown weird distribution"? I am asking because I saw some sources on the internet that say that Monte Carlo is an approximation technique. $\endgroup$– floydCommented Sep 25, 2019 at 17:49
-
$\begingroup$ Monte Carlo is an approximation technique that does not necessarily require simulating from a non-standard distribution. Take the example of importance sampling. $\endgroup$– Xi'anCommented Sep 25, 2019 at 18:08