# How can I sample from the conditional distribution?

I am learning Gibbs Sampling, in which there is a step named sampling from conditional distributions. I don't understand: 1. where is the conditional distribution from? From a general case, how can I get the conditional distribution?

Any help will be greatly appreciated.

• You get the conditional distribution from your model. – Glen_b Aug 6 '13 at 10:01
• Before trying to understand Gibbs sampling, you should learn the basics of probability theory and read up on conditional probability on your own. – Lucas Aug 6 '13 at 11:55

The goal is to generate samples for a multivariate distribution and its marginals, e.g. for the random vector $\mathbf{X} = (X_1, X_2, X_3)$, with pdf $f(x_1,x_2,x_3)$.
The conditional distributions, a.k.a. the full conditional distributions, are the distributions of each component random variable in the vector, conditional on the others, e.g. $X_1 | X_2=x_2, X_3=x_3$, and $X_2 | X_1=x_1, X_3=x_3$, and $X_3 | X_1=x_1, X_2=x_2$.
To get a full conditional, you treat the conditioning variables as constant in the multivariate pdf. E.g., to get the pdf for $X_1 | X_2=x_2, X_3=x_3$, you treat $x_2$ and $x_3$ as constant: the pdf $f(x_1|x_2,x_3)$ is obtained by simply starting with $f(x_1,x_2,x_3)$ and treating $x_2$ and $x_3$ as constant. What you hope is that you can recognize the core of the resulting pdf as a well-known univariate distribution, which you can easily sample from. Then, given a set of starting values, you sample from each full conditioning distribution in turn, using the most recent values of the conditioning variables.