Sampling from the conditional distribution assuming sampling from the joint

I am struggling with this question, which I thought it should be easy: suppose we have a method of sampling from the joint distribution of a collection of (discrete ordinal) random variables. We do not know the density. We need to find a way to sample from the conditional distribution of a subset of them, $X_i$, given specific values for the others, $Y_i = y_i$. Since they are discrete I thought of generating a very large sample from the joint and keep only those where $Y_i =y_i$. I am not sure if this makes sense and it also looks potentially extremely inefficient. Is there any other way to do this?

1) The method of sampling the joint and then looking at the sampling distribution of $$X$$ from the pairs where $$Y=y_i$$ will give you samples from the conditional distribution ($$p(X|Y=y_i)$$), no problem there.

2) there are a number of ways of generating conditionals but you usually need to know (at least!) their joint probability function or some other things you don't appear to know (the conditioning in the opposite direction plus the marginals for example).

Please note that for discrete variables, it's common not to call the probability function a density; that term more often applies to continuous random variables.

• Hi @Glen_b, thanks for the answer and the comment about the misuse of the term "density". No, unfortunately I do not know the probability function, I just have a method for generating samples. My concern with the method I mentioned is that the probability of $Y=y$ will be very very small, so I need to generate a huge amount of samples from the joint. Mar 21, 2013 at 16:18
• Hi @user20780. When you sample from the joint distribution of $X$ and $Y$, and look only at the values of $Y$, you have a sample from the marginal distribution of $Y$. I know that you know that. Now, if you do this, what is the Monte Carlo value that you get for $P(Y=y)$ for the particular $y$ you are interested in? You've said that it is very very small. Can you give us an idea of how small it is? Thanks.
– Zen
Mar 21, 2013 at 18:12
• Can you also give us an idea of the support of the discrete random variables $X$ and $Y$?
– Zen
Mar 21, 2013 at 18:24
• Hi @Zen, $P(Y=y)$ could be 1/1000 or less. The variables are ordinal with 3 to 5 levels each. Thanks a lot for your comments. Mar 21, 2013 at 18:32