Suppose that one investigates the two sources of variability with data $y_{ij}$ acquired from $j$th subject under $i$th session ($i=1,2; j=1,2..., n$). A linear mixed-effects model can be formulated as follows,

$$y_{ij} = \alpha_0 + b_i + c_j + \epsilon_{ij}$$

where $\alpha_0$ is a constant, $b_i$ and $c_j$ are the random effects for $i$th session and $j$th subject respectively, and $\epsilon_{ij}$ is a residual term. With assumptions of $b_i ~\sim N(0, \tau_1^2)$, $c_j ~\sim N(0, \tau_2^2)$, and $\epsilon_{ij} ~\sim N(0, \sigma^2)$, the intraclass correlation (ICC) values for sessions and subjects can be defined respectively as

$$ICC_{session} = \frac{\tau_1^2}{\tau_1^2+\tau_2^2+\sigma^2}, ICC_{subject} = \frac{\tau_2^2}{\tau_1^2+\tau_2^2+\sigma^2}.$$

And the above ICC values can be obtained through lmer in R package lme4. I have two questions:

  1. Is there a way to test the significance of the above ICC values in the context of LME model, similar to the $F$-stat for ICC defined under random-effects ANOVA?

  2. Intuitively the bigger the variability between the two sessions (more different between the two sessions), the higher the session ICC. But how is this intuition consistent with the notion that ICC measures reliability, reproducibility, or consistency? Is the session reliability (or reproducibility, consistency) about any two subjects' responses coming from the same session, not about the difference between the two sessions?

  • $\begingroup$ You can test those ICCs by simply testing the respective variance components in the numerator (i.e., $\tau^2_1$ and $\tau^2_2$). In other words, fit two models where you leave out one of the two components and then compare those reduced models against the full model using the anova function. $\endgroup$ – Wolfgang Dec 17 '13 at 23:24

While still waiting for any suggestions for the first question, I will try to answer the second question myself.

The correlation between two responses from two subjects $j_1$ and $j_2$ during the $i$th session is

$\frac{cov(y_{ij_1}, y_{ij_2})}{\sqrt{var(y_{ij_1})var(y_{ij_2})}}=\frac{E(b_i+c_{J_1}+\epsilon_{ij_1})E(b_i+c_{j_2}+\epsilon_{ij_2})}{\sqrt{var(y_{ij_1})var(y_{ij_2})}}=\frac{\tau_1^2}{\tau_1^2+\tau_2^2+\sigma^2}$

which is exactly $ICC_{session}$. That is, $ICC_{session}$ is the correlation between any two subjects' responses coming from the same session, and it's not about the difference between the two sessions.

Am I interpreting it correctly?

  • 1
    $\begingroup$ Your equation for the correlation is not correct. It should be the covariance divided by the two standard deviations (not the variances). Your end result is correct though. $\endgroup$ – Wolfgang Dec 18 '13 at 16:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.