It is suggested that MCMC can be used to simulated draws from a distribution which has a complex normalising constant $C$, so that we don't need to derive them (as an example, let us suggest Beta distribution $f(x) = C x^{a-1} (1-x)^{b-1}$). We cannot calculate constant $C$ so we perform draws form a Markov chain with transition matrix provided by i.e. a Metropolis-Hastings scheme. Now, after say 1000 draws, we will have something like a histogram of visited states, which will approximate the Beta(a, b) distribution.

BUT what is the point? We cannot simulate separate values (because states of the Markov chain are dependent, so we cannot assume that a state taken at some random time instant will be ~ Beta(a, b), but only as time -> infinity will the collective number of draws approximate the distribution in question).


In complement to Taylor's fine answer, let me signal that a Markov chain that is not strongly irreducible, that is, a chain that can only move to a limited number or range of values as in the "3 is followed by 2 or 4" example in the comments, can be turned into one by random (e.g., Poisson) subsampling:

Given a transition kernel or matrix K, at iteration $t$, generate $X_{t+1}$ as$$Y \sim K^\kappa(X_{t},\cdot),\qquad\kappa\sim\mathcal{P}(\lambda),\quad\lambda>0$$ as this makes the Markov chain strongly irreducible, i.e., able to go from anywhere to anywhere.(The associated Markov kernel is called the skeleton in Meyn and Tweedie (1993).)

  • $\begingroup$ Ok, this is the thing i've been looking for I think. I had no idea that irreducibility can be strong. $\endgroup$ – SWIM S. Aug 27 '17 at 14:04

We cannot simulate separate values (because states of the Markov chain are dependent, so we cannot assume that a state taken at some random time instant will be ~ Beta(a, b),

This is not true. Once the chain has converged, or after a "burn-in" period, every draws' marginal distributions will be the target distribution (Beta(a,b) in this case). This is because the Markov chain is stationary. If $X^i \sim \pi(x^i)$, and the transition distribution is $f(x^{i+1}|x^i)$, then at the next step $X^{i+1} \sim \int f(x^{i+1}|x^i)\pi(x^i)dx^i = \pi(x^{i+1})$. The last equality is the definition of stationarity/invariance. And yes, the draws are dependent, but there is no contradiction here.

But then we can easily depict Beta(a, b) WITHOUT the constant C,

I'm not sure if I follow this, but I will remind you that if you perform Metropolis-Hastings, at every iteration when you calculate the acceptance ratio, you do not have to evaluate the target density. Only something proportional to it. This is because the ratio of the unnormalized densities is the same as the ratio of the normalized ones, and these ratios are required for calculating the acceptance probabilities of your proposals at each iteration.

calculate numerically its integral, then take a number of points, say, 100,000, and divide this quantity into separate bins that we would have formed by dividing the random variable support into equal bin ranges, and assigning each bin a share of points out of 100,000 (depending on the area of Beta(a, b) over the bin range as percentage).

When you "calculate numerically" an integral, you're taking a sample average using all of your samples. Say you are interested in estimating the mean of your Beta(a,b) distribution. Then you take $$ \frac{1}{10000}\sum_i x^i, $$ and this approximates the integral/expectation. The reason this works is because the Markov chain designed by the Metropolis-Hastings algorithm is irreducible, and has as its stationary distribution the thing you're interested in.

The separation of the support into bins is only done for plotting purposes.

Both methods do not allow us to generate 1 separate draw from the distribution in question (or there is a way?).

I don't think you are describing two techniques. You are referring to one technique, the MH algorithm, and describing two things that you may do with your samples (as far as I can tell).

  • $\begingroup$ I think that maybe the example with beta is not that good. But if we consider generating from discrete distribution, where the chain is modelled as birth-death chain (proposal matrix - it should be aperiodic and irreducible), when at state $x_i = 3$, the chain cannot propose state $x_{i+1} = 10$ (has links to 2 and 4 only), though in original distribution (let us approximate sth like binomial for example) you can get 10 after 3 (if you perform binomial experiment twice - first time you get 3, next time you get 10). $\endgroup$ – SWIM S. Aug 27 '17 at 13:16
  • $\begingroup$ I modified the question because I realized there was only 1 question, the rest was just an irrelevant train of thought of mine. $\endgroup$ – SWIM S. Aug 27 '17 at 13:45
  • 1
    $\begingroup$ @SWIMS. The issue about always getting a 10 after a 3 can be bypassed by a Poisson subsampling of the original Markov chain. $\endgroup$ – Xi'an Aug 27 '17 at 13:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.