Drawing from a conditional density

Question

I have a simple question. Suppose $X=(X_1,X_2,X_3)$ is multivariate normal. What's the best (quickest) way to draw from the conditional density $X_1\mid \exp(X_1)+\exp(X_2)+X_3$?

Answer 1

Define a Markov chain like this.

1. Start with any point $x=(x_1,x_2,x_3)\in\mathbb{R}^3$ that satisfies $e^{x_1}+e^{x_2}+x_3=r$.

2. Draw independent $e_i\sim N(0,\epsilon_i)$, and define $z_i=x_i+e_i$, for $i=1,2,3$.

3. Define $$ (u_1,u_2,u_3)=\left( \frac{r\,e^{z_1}}{e^{z_1}+e^{z_2}+z_3}, \frac{r\,e^{z_2}}{e^{z_1}+e^{z_2}+z_3}, \frac{r\,z_3}{e^{z_1}+e^{z_2}+z_3}\right) \, . $$

4. Define $y=(y_1,y_2,y_3)=(\log u_1,\log u_2, u_3)$. It's clear that $e^{y_1}+e^{y_2}+y_3=r$.

5. Accept this proposal point $y$ as the next value of the chain with probability $\min \left\{ \frac{f(y)}{f(x)},1 \right\}$, where $f$ is the multivariate normal density, or reject the proposal and keep the old value of $x$ as the next value of the chain.

6. Repeat from step 2 with the new value of the chain playing the role of $x$.

The first coordinate of each value of this Markov chain will have as equilibrium distribution exactly what you need. You should discard the first, say, $5000$ thousand values of the chain, which correspond to the so called burn-in period. You have to find small values for the $\epsilon_i$'s that make the Markov chain mix properly. Large values of the $\epsilon_i$'s will make you reject all the proposals. My sugestion is to target an acceptance rate of about $50\%$ and do a graphical monitoring of the chain.

I've just seen on the bookstore the book "Monte Carlo Statistical Methods" by Robert and Casella that explain all you need to know to justify this Random Walk Metropolis Algorithm.

Good luck!

P.S. As clarified latter by cardinal, Prof. Robert is our fellow user Xi'an, who posted the first answer to this question!