Suppose I have parameters $\theta_0$ and $\theta_1$ with prior $$ p(\theta_0,\theta_1)=p(\theta_0|\theta_0<\theta_1)p(\theta_1),$$
that is, $\theta_0$ is less than $\theta_1$. The distributions are assumed to be continuous on their support. Apart from the likelihood, there are no other terms involving $\theta_0$ and $\theta_1$. I'm trying to consider potential sampling procedures for these parameters.
My current MCMC procedure is:
- Propose a value for $\theta_1$ using a random walk update.
- Use the independence sampler to sample a value of $\theta_0$ that is restricted to be less than the proposed value of $\theta_1$ using the appropriate truncated density (with the correction).
- Evaluate Metropolis-Hastings accept/reject.
Does anyone have ideas for other proposals here? The trick is that the support of $\theta_0$ changes, so a random-walk Metropolis step from the previous value of $\theta_0$ isn't guaranteed to fall in the support of $\theta_0$ given an updated value of $\theta_1$, so this set of draws would be rejected in a random-walk Metropolis scheme. This results in very few accepted proposals and increases the number of iterations needed.
My question is similar to a question posited here that was never answered.