Suppose we have a Metropolis-Hastings sampler for a target distribution $f$, and we use a proposal density $Q_t$, that may depend on time $t$. By construction, $f$ is still an invariant density of the chain, at every time step (*). However, the chain is not a Markov chain anymore, at least not a time homogeneous one. I'm pretty sure there exists some theory that ensures that the process still converges to $f$, but can not find it in standard references. Where can such results be found?

* We assume that basic non degeneracy conditions hold, for instance both $f$ and all $Q_t$ are non-negative on the whole state space.

  • $\begingroup$ Are the $Q_t$s fixed beforehand or do they depend on the history of the chain? $\endgroup$ – Juho Kokkala May 12 '17 at 6:00
  • $\begingroup$ I'm curious about both cases. But the particular $Q_t$ 'sI have depend on the history. $\endgroup$ – komark May 12 '17 at 10:09

What you are describing is adaptive MCMC if your proposal distribution depends on the history of the chain, and is thus time dependent. There is a lot of theory about ergodicity of adaptive mcmc. There are essentially two main conditions:

  1. Diminishing Adaptation: The adaptation on the proposal distribution should diminish as a function of $t$.

  2. Containment: The time to stationarity remains bounded in probability. This is more of a technical condition and often difficult to check.

You can find a variety of references on this:

These and these set of slides go into some detail about adaptive MCMC.

This is the paper that obtains the above conditions.

Here are some examples of adaptive MCMC.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.