# How is MCMC relevant for Bayesian inference? [duplicate]

I understand how MCMC can be useful for approximating the shape of a density function if you're able to generate random samples from that density function. However, I don't understand how this is applied towards Bayesian inference because any specific selection of model parameters using MCMC would yield a number, not a distribution.

For example, if I have a binomial likelihood function and a beta prior, I might use Metropolis-Hastings to select a random starting parameter, $$\theta=.3$$. When that starting parameter and some data are input into my likelihood function and prior, I get a deterministic number as the output. No randomness involved.

Framing this issue another way: Why does the MH algorithm generate the posterior distribution from the count of visits to a parameter value rather than using the posterior values generated directly?

• See this Q&A for use of a Gibbs Sampler to find a Bayesian posterior distribution. // Also see links under 'Related" in the righ-hand column of this page for other discussions of MCMC. Apr 27, 2020 at 1:24

With Monte Carlo methods you can approximate integrals that are otherwise very difficult to compute analytically. At each step of a Bayesian procedure you may need to normalize the product of the likelihood times the prior by computing an integral.

Example of a basic integration of x^2 in the interval [5,12]. The exact result is 1385/3 = 461.66. You can sample with uniform distribution in [5,12]. You obtain a MC approximation of the integral by computing the mean of the squares of the samples multiplied by 5. More simulations give better results.

a=runif(100)*5+7
mean(a^2)*5
[1] 485.4334

a=runif(1000)*5+7
mean(a^2)*5
[1] 462.2768

a=runif(10^8)*5+7
mean(a^2)*5
[1] 461.6724


Monte Carlo methods are also used to summarize the posterior distribution by sampling from it and obtaining highest posterior density regions, mean, mode...

• I appreciate the comment but I don't think you responded to the question in the post (as opposed to the title) Apr 27, 2020 at 3:04