Inter-(-intra) class correlation coefficient or intra-(inter-) concordance coefficient in python

Question

My question is about how to calculate inter-(intra-) class correlation coefficient (ICC) or intra-(inter-) concordance coefficient (CCC), ideally in python. My dataset consists of several dozen subjects. For each subject a feature was calculated using three different algorithms and each algorithm was repeated three times. So, I have 3x3=9 measurements for each subject. There is a lot of information on this forum about ICCs. But it is not clear to me which ICC I have to use. Ideally, I would like to make these calculations in python. Are there any python libraries for this?

Answer 1

I see no one has answered in 3 years and question may not be relevant.

But I would like to provide answer for future users.

Firstly, in Python you can use pingouin package:

import pandas as pd

#create DataFrame
df = pd.DataFrame({'exam': [1, 2, 3, 4, 5, 6, 1, 2, 3, 4, 5, 6,
                            1, 2, 3, 4, 5, 6, 1, 2, 3, 4, 5, 6],
                   'judge': ['A', 'A', 'A', 'A', 'A', 'A',
                             'B', 'B', 'B', 'B', 'B', 'B',
                             'C', 'C', 'C', 'C', 'C', 'C',
                             'D', 'D', 'D', 'D', 'D', 'D'],
                   'rating': [1, 1, 3, 6, 6, 7, 2, 3, 8, 4, 5, 5,
                              0, 4, 1, 5, 5, 6, 1, 2, 3, 3, 6, 4]})

import pingouin as pg

#https://pingouin-stats.org/generated/pingouin.intraclass_corr.html
icc = pg.intraclass_corr(data=df, targets='exam', raters='judge', ratings='rating')

icc.set_index('Type')

It returns a dataframe with lots of different ICC computed (ICC1, ICC2, ICC2). Just read the docs and choose the right one.

Secondly, you can also do the same thing in R

library(irr)
data1 <- data.frame(A=c(1, 1, 3, 6, 6, 7, 8, 9, 8, 7),
                   B=c(2, 3, 8, 4, 5, 5, 7, 9, 8, 8),
                   C=c(0, 4, 1, 5, 5, 6, 6, 9, 8, 8),
                   D=c(1, 2, 3, 3, 6, 4, 6, 8, 8, 9))

icc(data1, model = "oneway", type = "consistency", unit = "single")

Don't forget to read the package docs.