4
$\begingroup$

Trying to wrap my mind around Gibbs Sampling. Across many answers in this same forum, I constantly notice that the examples shown do not actually require an observed data set (First example (with R code); The D&D example*), the same for other sources in the web that try to explain.

Whereas in every equation there is always the conditional on y component:

Joint distribution as function of conditionals

  • As a secondary auxiliary question, I see the D&D example introduces an Accept-Reject algorithm, whereas in most other sources I just see that there is a sampling done from the conditional with no extra step (e.g. here "[Gibbs Sampling] accepts all proposals"). Is there something else I'm missing?
Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ The accept-reject algorithm in Gibbs sampling is normally the uniform distribution, so you see no extra step. You may need to know the MH Algorithm first. $\endgroup$
    – Lerner Zhang
    Commented Jun 19, 2019 at 23:03
  • $\begingroup$ The images attached come from course notes, but an equivalent notation is given in Regression (Fahrmeir, Kneib, Lang & Marx, 2013, p. 675): "In most cases no (simple) methods for directly drawing random numbers from the density p(theta|y) of the entire parameter vector are available. Often, however, random numbers can be directly drawn from the conditional densities p(theta_1 | ·), ... ,p(theta_s | ·), where p(theta_s | ·) denotes the conditional density of theta_s given all other blocks theta1, ... theta_s-1, theta_s+1, ... theta_s and the data y." $\endgroup$
    – Kuku
    Commented Jun 19, 2019 at 23:13

1 Answer 1

Reset to default
3
$\begingroup$

You are right - the conditioning data is not necessary: The Gibbs sampler is an MCMC method designed to sample from an arbitrary joint distribution, in cases where it is simpler to get the conditional distribution of each element (conditional on the other elements) than it is so get the marginal distribution of the elements. The Gibbs sampler is applicable when you are seeking to sample from a joint distribution that is conditional on some data (i.e., a joint posterior or predictive distribution); it is also applicable when there is no (explicit) conditioning variable.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Oh ok I see. To make it even more clear, the conditional distributions are derived analytically and the dataset has no influence on them? On further exploration I did find some conditional distributions that were also a function of y bar or n, in the latter case the our sample size would be needed to do the Gibbs sampling, right? $\endgroup$
    – Kuku
    Commented Jun 20, 2019 at 16:02
  • 1
    $\begingroup$ In any case where you are using the conditional distribution, given the data, that data will show up as part of the function (through the sufficient statistic). So yes, this might entail the conditional distributions being functions of $\bar{y}$ and $n$ in some cases. $\endgroup$
    – Ben
    Commented Jun 20, 2019 at 23:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.