Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I used the effectiveSize from the coda package to compute the effective sample size, but I get an effective sample size which is actually greater than my actual sample size (e.g. 11'043 to 10'000).
I am also aware of this and this thread, and also tried to use the ess command from the mcmcse package, but the results get even worse with this package.
Is there a workaround for that?
PS: If I should have posted this as answer to one of the other threads, please merge.
If the estimated effective sample size is larger than the number of samples, this likely means that there is negative estimated autocorrelation in your samples. If you run an acf on your samples, you will most certainly find negative correlation.
Remember that "effective sample size" is used as a way to compare variance of estimators obtained via correlated sampling to if independent samples had been obtained. Thus, if there is negative correlation in the samples, the variance of the estimator from correlated samples can be smaller than the variance of the estimator from independent samples. This in turn leads to larger effective sample size.
For theoretical detail, consider $X_1, X_2, \dots, X_n$ being samples from a distribution with mean $\mu$ and variance $\lambda^2$. Consider estimating $\mu$ with the sample mean.
If $X_i$ are independent, then variance of the sample mean $\lambda^2/n$.
If $X_i$ are correlated (say in MCMC), then (large sample) variance of the sample mean (using a CLT argument) is $\sigma^2/n$ where $$\sigma^2 = \lambda^2 + 2\sum_{k=1}^{\infty}\text{Cov}(X_1, X_{1+k}) \,. $$
Effective sample size is defined as $$ESS = n \dfrac{\lambda^2}{\sigma^2} \,.$$
Usually in MCMC, the samples are positive correlation, which means $\sigma^2 > \lambda^2$, but if the samples are negatively correlated, then $\sigma^2 < \lambda^2$, thus making $ESS$ larger than $n$.
The fact that using ess from mcmcse leads to an even higher ESS is not necessarily "worse". It is just giving you a better estimate of the ESS, since it estimates $\sigma^2$ directly. coda approximates $\sigma^2$ by fitting an ar1 process to the MCMC samples.
