I have two models (which I'm estimating by MCMC with Stan). There are more parameters in reality, but a simplified example is:
A: y ~ (1|group)
B: y ~ X + (1|group)
I then calculate the ICC in each model from the parameter chains.
I'd like to be able to say if the ICC is meaningfully lower in model B vs model A.... that is, that does X explain some of the variance attributable to group?
Is it reasonable to:
- Make 2 sets of draws for the ICC (one from each model).
- Sort each list and rank them
- Compute the difference for between draws with the same rank, and summarise this distribution (e.g. as if it were a parameter posterior)
This seems like it summarises the information from the models which is relevant to my question, but it feels hacky, and I'm worried that comparing posterior distributions in this way doesn't actually have the interpretation I'd like. Any thoughts on this much appreciated.