Stability of cross-validation in Bayesian models

Question

I'm fitting a Bayesian HLM in JAGS using k-fold cross-validation (k=5). I'd like to know whether estimates of parameter $\beta$ are stable across all folds. What's the best way to do this?

One idea is to find the differences of the posteriors of $\beta$ and to see if 0 is in the 95% CI of the difference. In other words, is 0 in the 95% interval of $\beta_{k=1}-\beta_{k=2}$ (and then repeat for all pairs of folds).

Another idea is to treat the posteriors from each fold as different MCMC chains, and to compute Gelman's Rhat (Potential Scale Reduction Factor) across these pseudo-chains.

Is one of these preferable, and are there alternatives?

Answer 1

It may not be a complete answer, but if 0 is NOT in the 95% CI for several differences it is quite safe to say that they are not identical at a 0.05 level.