1
$\begingroup$

When I learned multilevel modeling, one of the first things we were required to do is to run an empty model with the dependent variable and the clustering IDs. In my understanding, the ICC of such a model shows what percentage of the variance in the dependent variable is due to first and second level factors. I often relied on this method to decide if it makes sense to add an additional level to the analysis. As it's also indicated here, I thought the ICC of the empty model can be interpreted as a variance partition coefficient.

Recently I came across an example which made me question this interpretation. The units of analysis are political parties, observed across several elections which take place in multiple countries. The ICC of a two level model where parties are nested in elections is .68. The ICC of a two level model where parties are nested in countries is .53.

Theoretically it would make sense to run a three-level model where parties are nested in elections which are nested in countries. However, if I run such a model, the icc on the second, election|country level is .68 and on the third, country level is .47. My interpretation would be that this result warrants a three level model since a substantial portion of the variance is on these upper levels. However, what exactly is substantial? I always interpreted the ICC in percentage points. Which would mean 68 percent of the variance in the dependent variable is due to differences between elections and 47 percent of the variance in the dependent variable is due to differences between elections. However, the two do not add up to 1 or 100. Why is that? How should I think about it? Maybe I was always misinterpreting this, in which case I am looking for a correct way to read the ICC. Any feedback is welcome!

PS: Software-wise I usually do this in Stata. This would mean a code which looks similar to:

mixed dependent_var ||country: ||election:

in stata the third level comes before the second. After the model I type:

estat icc
$\endgroup$
1
$\begingroup$

I think parts of your question are answered here: Calculating covariance and ICC in mixed models?

you have two intra-correlation coefficients depending on which level you're looking at.

However, you can calculate an ICC, which takes all source of uncertainty into account and which computes the mean random effect variances (see Johnson PC, O'Hara RB. 2014. Extension of Nakagawa & Schielzeth's R2GLMM to random slopes models. Methods Ecol Evol, 5: 944-946. (doi: 10.1111/2041-210X.12225)).

There's an R implementation in sjstats::icc().

$\endgroup$
1
  • $\begingroup$ Thank you! This is indeed what I wanted. I am wondering what is going on with stata and why do I don't get the same values when I run the three level model. $\endgroup$
    – eborbath
    Nov 26 '18 at 23:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.