Gibbs sampling how to sample from the conditional probability? Bayesian model I want to learn Gibbs sampling for a Bayesian model. How can I sample the variable from the conditional distribution?

 In this example, arrow means dependent; for example, Grade depends on Difficulty and Intelligence.
To use Gibbs sampling to calculate the joint distribution, first I set the Difficulty and Intelligence to (1,1).
The next step is to sample Grade from the $\rm{P(Grade|Difficulty=1,Intelligence=1)}$, but how can I sample?
 A: Since we are calculating the joint distribution, we'll assume that our initial sample is $ x = P(D=0,I=0,G=0,L=0,S=0) $ .
To calculate the next sample, we'll need to sample each variable from the conditional distribution. 


*

*$ P(D\mid G,I,S,L) $,from the conditional independencies in the Bayes net, simplifies to just sampling $ P(D)$. We sample and get the value $D=1$.

*Similarly for $ I $, we sample and get the value $ I=1$.

*Sampling for $ P(G\mid D,I,S,L) $, due to the conditional independencies encoded by the Bayes net, simplifies to $ P(G\mid D,I) $. Since we have already sampled $ D=1,I=1 $, we use those values and sample $ P(G\mid D=1,I=1) $. In the CPD for Grade, we can choose one of the value from the last row (where $ D=1,I=1 $). We sample and get the value $ G=2 $ (the value 0.3)

*$ P(L\mid I, G,D,S) $ simplifies to $ P(L\mid G) $. We sample from the second row the Letter CPD, where $ G=2 $, and we sample and get $ L=1 $ (the value 0.6).

*Similarly, sample $ P(S \mid I,L, G,D,) $ by simplifying to $ P(S \mid I) $. We get $ S=1 $ (sampling from the second row of the CPD where $ I=1 $.


And we'll have a new sample $ x': P(D=1,I=1,G=2,L=1,S=1) $.
A: If the conditional distribution cannot be generated directly from standard random generators, you can apply a Metroplolis-Hastings schema within the Gibbs sampler.


*

*draw one sample, $D_t$, from $P(D|G_{t-1},I_{t-1},S_{t-1},L_{t-1})$ using Metropolis sampler with initial value $D_{t-1}$

*draw one sample, $G_t$, from $P(G|D_{t},I_{t-1},S_{t-1},L_{t-1})$ using Metropolis sampler with initial value $G_{t-1}$

*draw one sample, $I_t$, from $P(I|D_{t},G_{t},S_{t-1},L_{t-1})$ using Metropolis sampler with initial value $I_{t-1}$


...
reference: Introduction Monte Carlo Methods with R chapter 7
