I am currently implementing a multivariate random walk Metropolis sampler (Metropolis within Gibbs). One problem I have is that computing the likelihood is computationally expensive. Thus, I am updating multiple variables in a single Metropolis updating step and then do the accept-reject step.

Unfortunately, it seems that the variance of the different parameters I update in each step is very different, so it is hard to get the stepsize correct during adaptation. Choosing the same stepsize for all variables leads to very slow mixing of the chains, as the variables with larger variance are updated to slow. So I need to find a method to determine individual stepsizes.

I am currently thinking about the following update procedure:

  • Let $X=(X_1,\ldots,X_N)$ be the vector to be sampled.

  • Choose a subset of $n<N$ of the variables $\{X_{i_1},\ldots,X_{i_n}\}$; ($i_j\neq i_k$ if $j\neq k$).

  • Update the $X_{i_j}$ from the proposal and leave all others the same

  • Accept-reject based on the updated variables

The reason for this scheme is, that this would allow me (during adaptation) to only adapt the stepsizes of the actuall sampled variables and leave the others the same. Since the subset changes each time, this would allow me to get individual stepsizes for each variable.

However, I am not sure, if this method would ensure ergodicity, or if this would break the Gibbs sampler.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.