0
$\begingroup$

I came across the intraclass coefficient in quite a few papers discussing the reliability of instruments for clinical purposes. Some articles online seem to be advising to use this over the pearson coefficient. I did try to compare a set of two consecutive measurements with the same instrument on the same panel with the ICC and with Pearson. The outcome were similar if not identical so I was wondering : in which situation the use of the ICC would stands out compared to correlations with the Pearson coefficient when it comes to asses the reliability of an instrument?

Thanks in advance for any contribution.

$\endgroup$
1
$\begingroup$

Pearson's $r$ is a linearity index that quantifies how well two variables $x$ and $y$ are related by the following equation: $y=mx+b$. In contrast, the consistency ICC is an additivity index that quantifies how well $x$ and $y$ are related by $y=x+b$. Finally, the agreement ICC is an agreement index that quantifies how well $x$ and $y$ are related by $y=x$ (see McGraw & Wong, 1996).

If you apply these measures to inter-rater reliability as in your question, then $x$ and $y$ are two different raters' ratings of the same objects. So Pearson's $r$ will allow the raters to have their own means (with difference $b$) and for one rater's ratings to be multiplicatively higher or lower than the other rater's ratings (by a factor of $m$). Consistency ICC will allow the raters to have their own means but not to be multiplicatively different. Finally, agreement ICC will require the raters to provide the same ratings.

The choice of which measure to use depends on which behavior seems most reasonable for your purposes. If you are okay with the raters having their own means, then the consistency ICC is fine. But if you truly want the raters to be interchangeable, then the agreement ICC is probably best.

McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.