Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a simple question. Suppose $X=(X_1,X_2,X_3)$ is multivariate normal. What's the best (quickest) way to draw from the conditional density $X_1\mid \exp(X_1)+\exp(X_2)+X_3$?

share|improve this question
This is not a simple question. Have you derived the conditional density by the Jacobian formula? – Xi'an Nov 26 '11 at 7:08

Define a Markov chain like this.

  1. Start with any point $x=(x_1,x_2,x_3)\in\mathbb{R}^3$ that satisfies $e^{x_1}+e^{x_2}+x_3=r$.

  2. Draw independent $e_i\sim N(0,\epsilon_i)$, and define $z_i=x_i+e_i$, for $i=1,2,3$.

  3. Define $$ (u_1,u_2,u_3)=\left( \frac{r\,e^{z_1}}{e^{z_1}+e^{z_2}+z_3}, \frac{r\,e^{z_2}}{e^{z_1}+e^{z_2}+z_3}, \frac{r\,z_3}{e^{z_1}+e^{z_2}+z_3}\right) \, . $$

  4. Define $y=(y_1,y_2,y_3)=(\log u_1,\log u_2, u_3)$. It's clear that $e^{y_1}+e^{y_2}+y_3=r$.

  5. Accept this proposal point $y$ as the next value of the chain with probability $\min \left\{ \frac{f(y)}{f(x)},1 \right\}$, where $f$ is the multivariate normal density, or reject the proposal and keep the old value of $x$ as the next value of the chain.

  6. Repeat from step 2 with the new value of the chain playing the role of $x$.

The first coordinate of each value of this Markov chain will have as equilibrium distribution exactly what you need. You should discard the first, say, $5000$ thousand values of the chain, which correspond to the so called burn-in period. You have to find small values for the $\epsilon_i$'s that make the Markov chain mix properly. Large values of the $\epsilon_i$'s will make you reject all the proposals. My sugestion is to target an acceptance rate of about $50\%$ and do a graphical monitoring of the chain.

I've just seen on the bookstore the book "Monte Carlo Statistical Methods" by Robert and Casella that explain all you need to know to justify this Random Walk Metropolis Algorithm.

Good luck!

P.S. As clarified latter by cardinal, Prof. Robert is our fellow user Xi'an, who posted the first answer to this question!

share|improve this answer
I've just seen on the bookstore a couple of amazing books on MCMC by some guy named Robert: Is this a joke? Here's a fun idea: Click on the user name of Xi'an (the first commenter to the OP in this thread) and follow the link to his blog. – cardinal Feb 27 '12 at 15:04
Cool! I didn't know that Xi'an = Prof. Robert.Thanks, cardinal! – Zen Feb 27 '12 at 16:42
You're welcome. The juxtaposition is great! I'm sure the honest commentary makes Xi'an (rightfully) feel good. – cardinal Feb 27 '12 at 17:15

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.