1
$\begingroup$

I have to calculate intraclass correlation to quantify intra and inter-reliability. However, I am not totally sure whether the classical two-way ICC is appropriate because of the correlation among sample units.

In the experiment, each patient has more than one vessel that will be evaluated by 2 readers. The sample unit is the vessel.

The classical two-way ICC is calculated based on the model: $$ y_{ij} = \mu + s_i + r_j + \epsilon_{ij} $$ where $$ s_i \stackrel{i.i.d.}{\sim} N(0, \sigma^2_s) \mbox{ is a random effect for the vessel i;}\\ r_j \stackrel{i.i.d.}{\sim} N(0, \sigma^2_r) \mbox{ is a random effect for the rater j;} \\ \epsilon_{ij} \stackrel{i.i.d.}{\sim} N(0, \sigma^2) \mbox{ is a random effect for the measurement ij,} $$ then $$ ICC = \frac{\sigma^2_s}{\sigma^2_s + \sigma^2_r + \sigma^2}. $$

However, the random effects $s_i$ are not independent since they are from the same patient. I think it is necessary to include a random effect for the patients. In this way, the random effects $s_i$ could be considered independent as described below:

$$ y_{kij} = \mu + p_k + s_i + r_j + \epsilon_{kij} $$ where $$ p_k \stackrel{i.i.d.}{\sim} N(0, \sigma^2_p) \mbox{ is a random effect for the patient k;}\\ s_i \stackrel{i.i.d.}{\sim} N(0, \sigma^2_s) \mbox{ is a random effect for the vessel i;}\\ r_j \stackrel{i.i.d.}{\sim} N(0, \sigma^2_r) \mbox{ is a random effect for the rater j;} \\ \epsilon_{kij} \stackrel{i.i.d.}{\sim} N(0, \sigma^2) \mbox{ is a random effect for the measurement kij,} $$ then $$ ICC = \frac{\sigma^2_s}{\sigma^2_p + \sigma^2_s + \sigma^2_r + \sigma^2}. $$

Does it make sense?

$\endgroup$
1
$\begingroup$

I think I found my answer going back to the ICC definition:

The ICC is defined as the correlation between two observations. The correlation between two images from the same patient is given by

$$ \begin{eqnarray} ICC &=& Corr(y_{kij}, y_{kij'}) \\ &=& \displaystyle\frac{Cov(\mu + p_k + s_i + r_j + \epsilon_{kij}, \mu + p_k + s_i + r_{j'} + \epsilon_{kij})}{\sqrt{Var(y_{kij})Var(y_{kij'})}} \\ &=& \displaystyle\frac{Cov(p_k, p_k) + Cov(s_i, s_i)}{Var(y_{kij})} \\ &=& \displaystyle\frac{\sigma^2_p + \sigma^2_s}{\sigma^2_p + \sigma^2_s + \sigma^2_r + \sigma^2}, \end{eqnarray} $$

which is the same as the classical two-way ICC. The only difference is that I would be partitioning $\sigma^2_s$ in two parts given by $\sigma^2_s$ and $\sigma^2_p$.

However, the 95% confidence interval would be different because requires the independence assumption of the observations. Bootstrap seems a solution. The package lme4 has a function bootMer() that samples the random effects which can be considered independent.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.