Numerical examples proving and disproving the optimal scaling heuristic by Roberts et al

Question

Let

• $d\in\mathbb N$ with $d>1$
• $\ell>0$
• $\sigma_d^2:=\frac{\ell^2}{d-1}$
• $f\in C^2(\mathbb R)$ be positive with $$\int f(x)\:{\rm d}x=1$$ and $g:=\ln f$
• $Q_d$ be a Markov kernel on $(\mathbb R^d,\mathcal B(\mathbb R^d))$ with $$Q_d(x,\;\cdot\;)=\mathcal N_d(x,\sigma_dI_d)\;\;\;\text{for all }x\in\mathbb R^d,$$ where $I_d$ denotes the $d$-dimensional unit matrix

Now, let $$\pi_d(x):=\prod_{i=1}^df(x_i)\;\;\;\text{for }x\in\mathbb R^d$$ and $\left(X^{(d)}_n\right)_{n\in\mathbb N_0}$ denote the Markov chain generated by the Metropolis-Hastings algorithm with proposal kernel $Q_d$ and target density $\pi_d$ (with respect to the $d$-dimensional Lebesuge measure $\lambda^d$). Moreover, let $$U^{(d)}_t:=\left(X^{(d)}_{\lfloor dt\rfloor}\right)_1\;\;\;\text{for }t\ge0.$$ In the paper Weak convergence and optimal scaling of random walk Metropolis algorithms, the authors show (assuming that $g$ is Lipschitz continuous and satisfies some moment conditions) that if $X^{(d)}_0$ is distributed according to $\pi_d$, then $U^{(d)}$ converges (in the Skorohod topology) as $d\to\infty$ to the solution $U$ of $${\rm d}U_t=\frac{h(\ell)}2g'(U_t){\rm d}t+\sqrt{h(\ell)}{\rm d}W_t,$$ where $W$ is a standard Brownian motion, with $U_0\sim f\lambda^1$.

I'm searching for three interesting examples for a numerical study. One example should satisfy all of the assumptions above, another example should be a target density $\pi_d$ which is not of the product form above, but numerically agree with the result of Roberts et al. and the last example should be a target density $\pi_d$ which is again not of the product form above and numerically disagrees with the result.

Unfortunately, I'm very bad in finding concrete examples, since I've got no experience in this field outside purely theoretically considerations.

Answer 1

One example should satisfy all of the assumptions above,

A standard choice here would be a $d$-dimensional target which has all coordinates drawn iid from $\mathcal{N}(0, 1)$.

another example should be a target density $\pi_d$ which is not of the product form above, but numerically agree with the result of Roberts et al. and

A standard choice here would be to sample $u_1, \ldots, u_d \sim^{iid} \mathcal{U} [0, 1]$, and consider the $d$-dimensional target with independent coordinates in which the $i^{th}$ coordinate is drawn from $\mathcal{N}(0, u_i)$.

the last example should be a target density $\pi_d$ which is again not of the product form above and numerically disagrees with the result.

A standard choice here would be to consider the deterministic sequence $v_i = 2^{-i}$, and consider the $d$-dimensional target with independent coordinates in which the $i^{th}$ coordinate is drawn from $\mathcal{N}(0, v_i)$.