Based on results of Null model, some scholars suggest variance attributed to group level and individual level is computed using variance component output, whereas other suggest going the standard deviation scores.

Is there consensus how to calculate ICC?

  • 2
    Welcome to the site. Could you clarify your question a little? What exactly do the different authors recommend? (Formulas would help, references would help, quotes would help). – Peter Flom Aug 13 '13 at 10:36
  • 2
    Hi Peter, Gelman and Hill (p. 448) suggest using std. deviation to calculate ICC; formula std. dev of intercept/(sd of intercept+sd of residual). While others, for example iub.edu/~statmath/stat/all/hlm/hlm.pdf suggest variance estimates; formula (var. estimate of intercept/(var. estimate of intercept+ var. estimate of residual)) – Viplav Babu Aug 13 '13 at 10:52
  • and if I follow Bliese formula (p. 43) hrs.mod<-aov(HRS~as.factor(GRP),data=bhr2000) i get different answer cran.r-project.org/doc/contrib/Bliese_Multilevel.pdf – Viplav Babu Aug 13 '13 at 10:58
up vote 9 down vote accepted

Yes, there is a consensus: you should use the variances, not the standard deviations, in computing the intra-class correlation (ICC).

The two-level random-intercept-only model is $$ y_{ij} = \beta_0 + u_{0j} + e_{ij}, $$ where the random intercepts $u_{0j}$ have variance $\sigma^2_{u_0}$ and the residuals $e_{ij}$ have variance $\sigma^2_e$.

Now, the correlation between two random variables $x$ and $y$ is defined as $$ corr = \frac{cov(x, y)}{\sqrt{var(x)var(y)}}. $$

So to find the formula for intra-class correlation, we use the correlation formula and let our two random variables be two observations drawn from the same $j$ group, $$ ICC = \frac{cov(\beta_0 + u_{0j} + e_{1j}, \beta_0 + u_{0j} + e_{2j})}{\sqrt{var(\beta_0 + u_{0j} + e_{1j})var(\beta_0 + u_{0j} + e_{2j})}}, $$ and if you simplify this using the definitions given above and the properties of variances/covariances, you end up with $$ ICC = \frac{\sigma^2_{u_0}}{\sigma^2_{u_0} + \sigma^2_e}. $$

So for the two-level random-intercept-only model, the intra-class correlation is given by the ratio of the random intercept variance to the total variance.

If you were to use the square roots of these variances (i.e., the standard deviations), then it might still be a somewhat informative summary of how much variability we have at different levels of the model, but it could no longer be interpreted as an intra-class correlation coefficient.

By the way, I looked up the page in Gelman & Hill (2007) that you mentioned (p. 448), and they clearly define the ICC in terms of variances, not standard deviations. So I think this whole question could be based on an accidental misreading of their chapter.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.