3
$\begingroup$

I have seen in some papers that they have reported ICC as well as Cronbach's alpha. Does Cronbach's alpha provide more information about the reliability of a measured variable (e.g. heart rate at two different occasions) that ICC cannot? I presumed that since Cronbach's alpha is usually higher than ICC why one should report it as well? Could someone elaborate on the difference between ICC and Cronbach's alpha?

Many thanks in advance

$\endgroup$
2
  • $\begingroup$ There are many formulations of ICC and one is equivalent to Cronbach's alpha when applied to the same data. So alpha is like a special case of ICC. $\endgroup$ Commented Sep 5, 2017 at 18:43
  • $\begingroup$ In SPSS when I use ICC(3,1) Two-way mixed single measures (Absolute agreement) the output is accompanied by a Cronbach's alpha value. Does it mean it has additional information which has to be reported? $\endgroup$
    – Remy
    Commented Sep 6, 2017 at 8:00

1 Answer 1

5
$\begingroup$

There are four main formulations of the two-way intraclass correlation coefficient with different names according to different conventions. Whether these are considered "mixed effects" or "random effects" models does not affect their calculation (it only affects their interpretation), so I will lump the two together here based on functional equivalence and ignore that they are different in theory.

  1. Single measures consistency ICC = ICC(2,1) consistency = ICC(3,1) consistency = ICC(C,1)
  2. Single measures agreement ICC = ICC(2,1) agreement = ICC(3,1) agreement = ICC(A,1)
  3. Average measures consistency ICC = ICC(2,k) consistency = ICC(3,k) consistency = ICC(C,k)
  4. Average measures agreement ICC = ICC(2,k) agreement = ICC(3,k) agreement = ICC(A,k)

The differences between these formulations are derived from whether they are estimating the reliability of a single measure (e.g., one item, rater, or measurement occasion) or the average of all measures (e.g., all items, raters, or occasions), and whether they are quantifying consistency (i.e., allowing each measure to have its own mean) or agreement (i.e., requiring all measures to have the same mean). All other things being equal, average measures ICCs tend to be higher than single measures ICCs, and consistency ICCs tend to be higher than agreement ICCs.

If you have to choose a single ICC formulation, pick the one that most closely matches how you will use the measure(s): if you will use a single measure, then use one of the single measures formulations but if you will use the average of all measures then use one of the average measures formulations. Similarly, if your application is okay with different measures having different means, then you can use one of the consistency formulations but if your application needs all measures to have the same mean then use one of the agreement formulations. In practice, however, you can calculate all four and learn more about your data by examining the extent to which they differ in magnitude.

Cronbach's alpha coefficient is functionally equivalent to the third formulation listed above: the average measures consistency formulation or ICC(C,k). If you don't believe me, ask SPSS to output the two-way consistency ICCs and compare the alpha value it gives you to the average measures ICC it gives you; they will be the same. Cronbach's alpha is probably included in the SPSS output alongside the ICC values because alpha is a popular measure in many fields; you definitely don't "have to" report both and, indeed, reporting it alongside ICC(C,k) would be redundant. I wouldn't rely on the SPSS defaults (in this case or many others) to guide you toward statistical "best practice."

References

McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46.

Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.