# Metropolis Hastings proposal for one parameter restricted to less than the other

Suppose I have parameters $$\theta_0$$ and $$\theta_1$$ with prior $$p(\theta_0,\theta_1)=p(\theta_0|\theta_0<\theta_1)p(\theta_1),$$

that is, $$\theta_0$$ is less than $$\theta_1$$. The distributions are assumed to be continuous on their support. Apart from the likelihood, there are no other terms involving $$\theta_0$$ and $$\theta_1$$. I'm trying to consider potential sampling procedures for these parameters.

My current MCMC procedure is:

1. Propose a value for $$\theta_1$$ using a random walk update.
2. Use the independence sampler to sample a value of $$\theta_0$$ that is restricted to be less than the proposed value of $$\theta_1$$ using the appropriate truncated density (with the correction).
3. Evaluate Metropolis-Hastings accept/reject.

Does anyone have ideas for other proposals here? The trick is that the support of $$\theta_0$$ changes, so a random-walk Metropolis step from the previous value of $$\theta_0$$ isn't guaranteed to fall in the support of $$\theta_0$$ given an updated value of $$\theta_1$$, so this set of draws would be rejected in a random-walk Metropolis scheme. This results in very few accepted proposals and increases the number of iterations needed.

My question is similar to a question posited here that was never answered.

• This answer, to a slightly different question, may be useful: stats.stackexchange.com/questions/73885/… – jbowman Aug 14 at 19:13
• @jbowman: Thanks, I'll check it out! – Alex Aug 14 at 21:47
• You could transform the parameters to $\theta_1, \delta$ where $\delta > 0$ and $\theta_0 = \theta_1 - \delta$. Then the constraint disappears and the support is the same at each iteration. – jbowman Aug 15 at 14:51

If the question is about simulating from a target with a density of the form$$f(\theta_0,\theta_1)\mathbb I_{\theta_0<\theta_1}\tag{1}$$a Metropolis-Hastings algorithm does not need to enforce the constraint (1) in the proposal as proposed values that do no satisfy the constraint (1) will be rejected at the accept-reject step.

A natural approach in this case is to use a Metropolis-within-Gibbs approach that

1. simulate $$\theta_1$$ conditional on $$\theta_0$$ and possibly the current value of $$\theta_1$$, $$\theta_1^-$$, from a proposal $$p_1(\theta_1|\theta_0, \theta_1^-)$$ and accept or reject through a Metropolis-Hastings step;
2. simulate $$\theta_0$$ conditional on $$\theta_1$$ and possibly the current value of $$\theta_0$$, $$\theta_0^-$$, from a proposal $$p_0(\theta_0|\theta_1, \theta_0^-)$$ and accept or reject through a Metropolis-Hastings step

This way, the constraint is taken into account symmetrically between $$\theta_0$$ and $$\theta_1$$.

• Thanks for the response. Could you please clarify how this would work in the following scenario? The current value of $\theta_0$ is $\theta_0^-$. Suppose after proposing and accepting $\theta_1^*$ a new value for $\theta_1$, $\theta_1^* < \theta_0^-$. Upon proposing a new value $\theta_0^*$ for $\theta_0$, the acceptance ratio is $$\frac{\pi(\theta_0^*, \theta_1^*)}{ \pi(\theta_0^-, \theta_1^*)}\frac{f(\theta_0^- | \theta_0^*)}{f(\theta_0^* | \theta_0^-)},$$ but the value of $\pi(\theta_0^-, \theta_1^*)$ will be zero because $\theta_0^-$ does not fall in the support determined by $\theta_1^*$. – Alex Aug 15 at 16:34
• I just thought through it. $\theta_1^*$ would never had been accepted in the first place because it was less than the current value of $\theta_0$ when it was drawn. – Alex Aug 15 at 18:35
• Indeed, at each step, the constraint cannot but hold. – Xi'an Aug 16 at 7:26
• I didn't read the paper I posted carefully enough--@Xi'an is right, this is the correct approach. – Sheridan Grant Aug 16 at 18:07