(For those interested, my MATLAB implementation of multiESS
is available here.)
I am re-implementing the multivariate Effective Sample Size (multiESS) estimator for MCMC from Vats et al. (2015). User @Greenparker gave an excellent description of the method in this answer; I copy here the relevant part since I couldn't do better (check her full answer for more details):
The multivariate ESS returns one number for the effective sample size for the quantities you want to estimate; and it does so by accounting for all the cross-correlations in the process. [...] Suppose you are interested in the $p$-vector of means of the posterior distribution. The mESS is defined as follows $$\text{mESS} = n \left(\dfrac{|\Lambda|}{|\Sigma|}\right)^{1/p}. $$ Here
- $\Lambda$ is the covariance structure of the posterior (also the asymptotic covariance in the CLT if you had independent samples)
- $\Sigma$ is the asymptotic covariance matrix in the Markov chain CLT (different from $\Lambda$ since samples are correlated.
- $p$ is number of quantities being estimated (or in this case, the dimension of the posterior.
- $|\cdot|$ is the determinant.
- [$n$ is the number of samples.]
mESS can be estimated by using the sample covariance matrix to estimate $\Lambda$ and the batch means covariance matrix to estimate $\Sigma$. This has been coded in the function
multiESS
in the R package mcmcse.
The only nontrivial part in the algorithm is the batch estimation of $\Sigma$, which requires a choice of the batch size $1 \le b_n \le n$.
The default choice for $b_n$ in multiESS
is $\lfloor n^{1/2} \rfloor$. However, the method to choose $b_n$ is the paper is not completely clear and it is somewhat arbitrary (e.g., the authors sometimes pick $\lfloor n^{1/3} \rfloor$). Also, the authors show that the choice of $b_n$ should depend on the mixing time of the Markov chain, with slow-mixing processes requiring a larger $b_n$.
In practice, the choice of $b_n$ can affect the output of $\text{mESS}$ by an order of magnitude or even more.
As a practical, conservative choice, I am thinking of using for analyses the lower bound $\text{mESS}^*$ defined as follows: $$ \text{mESS}^* \equiv \min_{b_\text{min} \le b_n \le b_\text{max}} \text{mESS}(b_n)$$ where $b_\text{min} \ge 1$ is a small number (e.g., $b_\text{min} \equiv \lfloor n^{1/4} \rfloor$) and $b_\text{max} \le n$ is taken such that there is a reasonable number of batches to average over (e.g., $b_\text{max} \equiv \lfloor n/10 \rfloor$).
Does this sound sensible? Any other suggestion on how to choose $b_n$ in an objective way?
(A quick test suggests that $\text{mESS}^*$ seems to be not as conservative as taking the minimum of the univariate ESS across dimensions; so it probably strikes a good balance.)