Suppose I have parameters $\theta_0$ and $\theta_1$ with prior $$ p(\theta_0,\theta_1)=p(\theta_0|\theta_0<\theta_1)p(\theta_1),$$

that is, $\theta_0$ is less than $\theta_1$. The distributions are assumed to be continuous on their support. Apart from the likelihood, there are no other terms involving $\theta_0$ and $\theta_1$. I'm trying to consider potential sampling procedures for these parameters.

My current MCMC procedure is:

  1. Propose a value for $\theta_1$ using a random walk update.
  2. Use the independence sampler to sample a value of $\theta_0$ that is restricted to be less than the proposed value of $\theta_1$ using the appropriate truncated density (with the correction).
  3. Evaluate Metropolis-Hastings accept/reject.

Does anyone have ideas for other proposals here? The trick is that the support of $\theta_0$ changes, so a random-walk Metropolis step from the previous value of $\theta_0$ isn't guaranteed to fall in the support of $\theta_0$ given an updated value of $\theta_1$, so this set of draws would be rejected in a random-walk Metropolis scheme. This results in very few accepted proposals and increases the number of iterations needed.

My question is similar to a question posited here that was never answered.

  • 1
    $\begingroup$ This answer, to a slightly different question, may be useful: stats.stackexchange.com/questions/73885/… $\endgroup$
    – jbowman
    Aug 14, 2019 at 19:13
  • $\begingroup$ @jbowman: Thanks, I'll check it out! $\endgroup$
    – Alex
    Aug 14, 2019 at 21:47
  • $\begingroup$ You could transform the parameters to $\theta_1, \delta$ where $\delta > 0$ and $\theta_0 = \theta_1 - \delta$. Then the constraint disappears and the support is the same at each iteration. $\endgroup$
    – jbowman
    Aug 15, 2019 at 14:51

1 Answer 1


If the question is about simulating from a target with a density of the form$$f(\theta_0,\theta_1)\mathbb I_{\theta_0<\theta_1}\tag{1}$$a Metropolis-Hastings algorithm does not need to enforce the constraint (1) in the proposal as proposed values that do no satisfy the constraint (1) will be rejected at the accept-reject step.

A natural approach in this case is to use a Metropolis-within-Gibbs approach that

  1. simulate $\theta_1$ conditional on $\theta_0$ and possibly the current value of $\theta_1$, $\theta_1^-$, from a proposal $p_1(\theta_1|\theta_0, \theta_1^-)$ and accept or reject through a Metropolis-Hastings step;
  2. simulate $\theta_0$ conditional on $\theta_1$ and possibly the current value of $\theta_0$, $\theta_0^-$, from a proposal $p_0(\theta_0|\theta_1, \theta_0^-)$ and accept or reject through a Metropolis-Hastings step

    This way, the constraint is taken into account symmetrically between $\theta_0$ and $\theta_1$.

  • $\begingroup$ Thanks for the response. Could you please clarify how this would work in the following scenario? The current value of $\theta_0$ is $\theta_0^-$. Suppose after proposing and accepting $\theta_1^*$ a new value for $\theta_1$, $\theta_1^* < \theta_0^-$. Upon proposing a new value $\theta_0^*$ for $\theta_0$, the acceptance ratio is $$\frac{\pi(\theta_0^*, \theta_1^*)}{ \pi(\theta_0^-, \theta_1^*)}\frac{f(\theta_0^- | \theta_0^*)}{f(\theta_0^* | \theta_0^-)},$$ but the value of $\pi(\theta_0^-, \theta_1^*)$ will be zero because $\theta_0^-$ does not fall in the support determined by $\theta_1^*$. $\endgroup$
    – Alex
    Aug 15, 2019 at 16:34
  • $\begingroup$ I just thought through it. $\theta_1^*$ would never had been accepted in the first place because it was less than the current value of $\theta_0$ when it was drawn. $\endgroup$
    – Alex
    Aug 15, 2019 at 18:35
  • $\begingroup$ Indeed, at each step, the constraint cannot but hold. $\endgroup$
    – Xi'an
    Aug 16, 2019 at 7:26
  • 1
    $\begingroup$ I didn't read the paper I posted carefully enough--@Xi'an is right, this is the correct approach. $\endgroup$ Aug 16, 2019 at 18:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.