2
$\begingroup$

I know that Monte Carlo is used to approximate an integral by sampling. I also learnt MCMC algorithms such as Metropolis-Hastings and Gibbs sampling but I don't know where the "Monte Carlo" part is in those algorithms. Those algorithms are just sampling.

$\endgroup$
  • 3
    $\begingroup$ en.wikipedia.org/wiki/Monte_Carlo_method Monte Carlo methods are just sampling, usually with the goal of using certain facts about how the sampling was conducted to draw inferences about a problem that would be challenging to solve directly. $\endgroup$ – Sycorax says Reinstate Monica Sep 25 '19 at 12:57
  • $\begingroup$ @Sycorax Thank you for the answer. There is only one thing that is confusing to me. Does "Monte Carlo" concern itself with "APPROXIMATING an integral (given that we know how to sample from the distribution)" or "SAMPLING from an unknown weird distribution"? I am asking because I saw some sources on the internet that say that Monte Carlo is an approximation technique. $\endgroup$ – floyd Sep 25 '19 at 17:52
  • $\begingroup$ Monte Carlo methods include both. On the page that I linked, the experiment about approximating $\pi$ can also be viewed as a MC method to approximate an integral (the area of the circular section). And one application of MC for Bayesian statistics is sampling from an inconvenient target distribution. $\endgroup$ – Sycorax says Reinstate Monica Sep 25 '19 at 18:53
5
$\begingroup$

enter image description here

As I explained earlier this week in my introductory lecture to Monte Carlo methods, the founding principle of such numerical methods is the Law of Large Numbers, or the stabilisation of empirical frequencies to their expectations. Markov chain Monte Carlo algorithms are a special case of methods implementing the Monte Carlo principle, in that Markov chains are created with the specific purpose of preserving the convergence of empirical averages, which is then called the ergodic theorem. And with the rationale that these Markov chains are easier to produce than iid simulations from the given target.

Further links on this forum on the meaning of Monte Carlo:

$\endgroup$
  • $\begingroup$ Thank you for the answer. There is only one thing that is confusing to me. Does "Monte Carlo" concern itself with "APPROXIMATING an integral (given that we know how to sample from the distribution)" or "SAMPLING from an unknown weird distribution"? I am asking because I saw some sources on the internet that say that Monte Carlo is an approximation technique. $\endgroup$ – floyd Sep 25 '19 at 17:49
  • $\begingroup$ Monte Carlo is an approximation technique that does not necessarily require simulating from a non-standard distribution. Take the example of importance sampling. $\endgroup$ – Xi'an Sep 25 '19 at 18:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.