MCMCglmm interpretation of effective sample size

Question

I'm running a selection of MCMCglmm models in R, and have a basic question about the outputs.

Based on what I've been reading, one indication that the model is mixing well is a large effective sample size. I'm finding this is true for most predictor/independent variables in the model -- with eff.samp ranging somewhere in the hundreds or thousands (my data set is based on about n = 800 dyads as data points).

However, for some of the terms I'm finding the effective sample size is extremely small - as in, around 10. I would think this is a sign to be suspicious of the output for these variables in particular, yet these variables often have very small p values (<.001 or even .000).

I've been unable to find a clear explanation of the effective sample size and its relationship to the posterior estimates and pMCMC values. What does a small effective sample size mean, in this type of model?

In case it's relevant, some background on the data: My data are hierarchically nested by dyad ID, and by individual #1 & #2 IDs, and also by geographic location (2 possible locations). I have one response variable and up to four predictor variables in this set of models. The response variable is count data and is zero-inflated, so I am using a zero-inflated poisson distribution.

Answer 1

Stan is a popular and powerful MCMC sampler and their documentation about diagnostics mentions:

In the plot, the points representing the values of $n_{\rm eff}/N$ are colored based on whether they are less than 0.1, between 0.1 and 0.5, or greater than 0.5.. These particular values are arbitrary in that they have no particular theoretical meaning, but a useful heuristic is to worry about any $n_{\rm eff}/N$ less than 0.1.

One important thing to keep in mind is that these ratios will depend not only on the model being fit but also on the particular MCMC algorithm used. One reason why we have such high ratios of $n_{\rm eff}/N$ is that the No-U-Turn sampler used by rstan generally produces draws from the posterior distribution with much lower autocorrelations compared to draws obtained using other MCMC algorithms (e.g., Gibbs).

Where $N$ is the total number of samples. So yes, a comparitively low $N_{\rm eff}$ is generally bad. I imagine your p-value is for whether the parameter's 95% excludes 0, but if you only have a small number of effective samples, you could have a deceivingly small range of values and this small range could give your p test overconfidence.