I wasn't able to find anything on google, but is there a variant of the Metroplis-Hastings algorithm where the acceptance probability (not the proposal kernel) in the $i$th iteration might depend on the stats $X_0,\ldots,X_{i-1}$ generated so far?
I guess the problem is that the resulting process is not a time-homogeneous Markov chain anymore. However, it might still obey a law of large numbers ...
In order to motivate what I would like to do: I would like to generate a fixed amount of candidate proposals in each iteration (like in Multiple-Try Metropolis-Hastings) and choose among the candidates the proposal which has the largest distance to all of the already generated states. Would this still tend (in a suitable sense) to the target distribution?
EDIT: Let me try to formalize this: Let
- $(E,\mathcal E,\lambda)$ be a measure space;
- $p:E\to[0,\infty)$ be $\mathcal E$ measurable with $$c:=\lambda p\in(0,\infty)$$ and $$\mu:=\frac{p\lambda}c;$$
- $k\in\mathbb N$;
- $q:E^k\times E\to[0,\infty)$ be $\mathcal E^{\otimes k}\otimes\mathcal E$-measurable with $$c_x:=\lambda q(x,\;\cdot\;)\in(0,\infty)\;\;\;\text{for all }x\in E^k$$ and $$Q(x,\;\cdot\;):=\frac{q(x,\;\cdot\;)\lambda}{c_x}\;\;\;\text{for }x\in E^k.$$
The idea would be that we are actually want to obtain a Markov chain $(X_n)_{n\in\mathbb N_0}$ with stationary distribution $\mu$ by running the Metropolis-Hastings algorithm with proposal kernel $Q$. By definition, $Q$ depends on the last $k$ states.
In order to actually apply this, we need to run the Metropolis-Hastings algorithm on $(E^k,\mathcal E^{\otimes k})$. I guess the target distribution should be $$\tilde \mu:=\tilde p\lambda^{\otimes k},$$ where $$\tilde p(x):=\prod_{i=1}^kp(x_i)\;\;\;\text{for }x\in E^k.$$ However, I Have no idea how I need to define the proposal kernel $\tilde Q$ on $(E^k,\mathcal E^{\otimes k})$, since it should intuitively be given by $$\tilde Q(x,B_1\times\cdots\times B_k)=\prod_{i=1}^{k-1}\delta_{x_i}(B_i)Q(x_1,\ldots,x_k,B_k);$$ which doesn't work since this proposal kernel doesn't admit a density with respect to $\lambda^{\otimes k}$.
(maybe we should replace the domain of $q$ by $\bigcup_{i=1}^kE_i\times E$)