1
$\begingroup$

In one article explaining MCMC, I once read the following statement.

The idea of sampling methods is the following. Let’s assume first that we have a way (MCMC) to draw samples from a probability distribution defined up to a factor. Then, instead of trying to deal with intractable computations involving the posterior, we can get samples from this distribution (using only the not normalised part definition) and use these samples to compute various punctual statistics such as mean and variance or even to approximate the distribution by Kernel Density Estimation.

Based on this above explanation, my understanding is that MCMC get samples from unnormalized distribution, but these samples can be used to compute the functional statistics of the corresponding normalized distribution. Is my understanding correct? How to prove the functional statistics calculated using samples from unnormalized distribution will be the same as the one from normalized distribution?

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ This question is about the Monte Carlo principle as a whole rather than directly about MCMC, which is one special case. $\endgroup$
    – Xi'an
    Commented Dec 7, 2023 at 4:14

2 Answers 2

Reset to default
0
$\begingroup$

The sampling algorithms are designed in such a way that frequency of samples is proportional to probability associated with respective state of parameter space. Then, if you for example want to calculate mean value of some parameter conditioned on specific value of some other parameter you average across relevant samples.

For proof that the sampling procedures generate appropriate sample distribution I recommend books by Bishop (Pattern recognition and machine learning) and MacKay's book on information theory.

Improve this answer
$\endgroup$
0
$\begingroup$

In Bayesian, in my view,

the MCMC is a procedure to draw samples from your priors (your assumptions) and follow your hierarchical model (also your assumptions), you can estimate the posterior distribution (the goal).

After performing diagnostics to your posterior, then you could verify whether all your assumptions make sense.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.