I'm interested in sampling a collection of variables with a sum constraint on them. For a simplified example:


$X \sim \mathcal{N}(0, 1)$

$Y \sim \mathcal{N}(0, 1)$


$X + Y = 1$

Proposed approach:

Use augmented variables $S$, $x'$, and $y'$, defined as:

$X = x' / S$

$Y = y' / S$

$S = x' + y'$

MCMC Algorithm:

1) Sample $x'$ given previous value of $S$ as being from $\mathcal{N}(0, S^2)$.

2) Sample $y'$ given previous value of $S$ as being from $\mathcal{N}(0, S^2)$.

3) "Sample" $S$ by setting it to $x' + y'$.

This feels like it should work, but how can I prove that it is a valid data augmentation scheme? Specifically, when sampling $x'$ using Gibbs sampling, why are we not forced to choose $x' = S - y'$ since we are conditioning on $S$ and $y'$?

  • 1
    $\begingroup$ The augmentation scheme does not work because your target distribution is not defined: you do not have a joint model on $(X,Y)$. $\endgroup$ – Xi'an Jan 11 '15 at 8:43

Sorry, I'm not going to answer your question directly. But, I think you are thinking about the problem in the wrong way.

If the sum of X and Y are 1, then it means that Y is completely determined by X. In that case, you can just sample X, and let Y to be 1-X. That will give you a random sample of X,Y with X+Y=1. The distribution of X completely determines the distribution of Y, so you only need to sample X.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.