I have a number of queries about computing the acceptance rate of samples generated from the Metropolis (symmetric random walk) algorithm empirically, that is, in the presence of burning-in and thinning of the autocorrelated samples.
To provide some detail to anchor the question, as part of an assignment for self-study, I am interested in sampling parameter vectors $\boldsymbol{\theta}$ from a posterior $p(\boldsymbol{\theta} | \mathbf{y})$ given data $\mathbf{y}$, using a symmetric proposal distribution, that is, an isotropic Gaussian proposal on the parameters $\boldsymbol{\theta}' \sim \mathcal{N}(\boldsymbol{\theta}' | \boldsymbol{\theta}, \sigma^2 \mathbf{I})$. I am also mandated to used one fixed initialisation of $\mathbf{0}$.
Now I am mandated to explore the behaviour of the Metropolis algorithm for various values of the isotropic Gaussian covariance parameter $\sigma^2$, and "thinning" parameter $t$. The thinning parameter $t$, as I understand, is a means of dealing with the autocorrelated samples generated from the Metropolis algorithm after burning it in for an appropriate number of iterations. I am required to report the empirical acceptance rate/rejection ratio for various values of the MCMC parameters $\sigma^2$ and $t$.
Given that I want to collect 5000 samples of the parameters, $\{ \boldsymbol{\theta}^{(n)} \}^N_{n=1}$ from the posterior I am mandated to do the following for say a thinning parameter of $t = 2$:
Run the Metropolis algorithm for $B$ burn-in iterations, yielding $B$ iterations of the parameter vector $\{ \boldsymbol{\theta}^{(b)} \}^B_{b=1}$. I will discard these samples.
Continue running the Metropolis algorithm for $T = 5000t$ iterations, where $t$ is the thinning parameter. For a thinning parameter of $t = 2$, that means I continue running the Metropolis algorithm for $T = 10000$ iterations, resulting in 10000 values of the parameter vector $\{ \boldsymbol{\theta}^{(s)} \}^T_{s=1}$
For the case where the thinning parameter is $t = 2$, I only collect samples every 2nd iteration out of a total 10,000 iterations above. This should generate $N = 5000$ samples from the posterior (hopefully, depending on a raft of other diagnostics), that is, $\{ \boldsymbol{\theta}^{(n)} \}^N_{n=1}$.
Query.
I now use the term acceptance rate to refer to the proportion of samples that are accepted by the Metropolis algorithm empiricallly, and not the quantity frequently denoted as $\alpha$ used to implement the decision rule for accepting/rejecting samples. Primarily, I am soliciting further clarity on how this metric is conventionally reported. Further, whilst I am aware that the empirical acceptance rate can be "tuned", that is currently outside the remit of my assignment.
Q1. In the case that the acceptance rate is supposed to be a single number, when I report the acceptance rate of samples from the Metropolis algorithm, do I report the acceptance rate computed using a total of $T$ autocorrelated samples, or do I compute it after I have adjusted for autocorrelation via thinning, that is, using a total of $N$ samples? Are there any conventions on this?
In the former case, that would mean using $\{ \boldsymbol{\theta}^{(s)} \}^T_{s=1}$ from step 2. to compute: $$\text{acceptance rate} = \frac{1}{T} \sum^T_{s=1} \mathbb{I}(\boldsymbol{\theta}^{(s)} \text{accepted})$$ In the latter case, that would amount to using $\{ \boldsymbol{\theta}^{(n)} \}^N_{n=1}$ from step 3. to compute: $$\text{acceptance rate} = \frac{1}{N} \sum^N_{n=1} \mathbb{I}(\boldsymbol{\theta}^{(n)} \text{accepted})$$
- Q2. Are there conventions whereby the acceptance rate is not reported as a single one-off number, but is rather time-varying? (I have read somewhere, I think in "Bayesian Data Analysis" or "Monte Carlo Statistical Methods" that a sliding window can be used.
Some assistance would be greatly appreciated.
