# Adjust intraclass correlation for measurement error

It's been known since Spearman (1904) that one can adjust an observed Pearson correlation for measurement error using this formula (R code with tidyverse attached):

true_r = observed_r / sqrt(rel_x * rel_y)


Thus, for instance, if we have an observed correlation of .4, and reliabilities of x and y of .6 and .8, we get:

true_r = 0.577 = .4 / sqrt(.6 * .8)


My question is: how does one do the equivalent correction for an intraclass correlation? I know, people normally use ICC as a measure of reliability, but I need to use it as a metric of within group similarity across raters.

Here things get complicated because there are multiple types of ICC, as well as single and average versions (following the taxonomy in psych package, at least). In my case, I am interested in the single rater case, type irrelevant (values are identical for my purposes).

Suppose we have 2 targets rated each 10 times. Suppose their true scores are 0 and -1, respectively. We can easily simulate some data for this and calculate the ICC values:

set.seed(1)
n = 10
testdf = data_frame(
a = rnorm(n),
b = rnorm(n, mean = -1)
)
testdf
# A tibble: 10 x 2
a       b
<dbl>   <dbl>
1 -0.626  0.512
2  0.184 -0.610
3 -0.836 -1.62
4  1.60  -3.21
5  0.330  0.125
6 -0.820 -1.04
7  0.487 -1.02
8  0.738 -0.0562
9  0.576 -0.179
10 -0.305 -0.406
psych::ICC(testdf %>% t())
Call: psych::ICC(x = testdf %>% t())

Intraclass correlation coefficients
type  ICC   F df1 df2     p lower bound upper bound
Single_raters_absolute   ICC1 0.26 4.5   1  18 0.049      -0.026           1
Single_random_raters     ICC2 0.26 4.5   1   9 0.064      -0.027           1
Single_fixed_raters      ICC3 0.26 4.5   1   9 0.064      -0.040           1
Average_raters_absolute ICC1k 0.78 4.5   1  18 0.049      -0.343           1
Average_random_raters   ICC2k 0.78 4.5   1   9 0.064      -0.355           1
Average_fixed_raters    ICC3k 0.78 4.5   1   9 0.064      -0.620           1

Number of subjects = 2     Number of Judges =  10


So, for single raters we get .26. The expected value is actually .33, which one sees if one uses a larger sample size:

n2 = 10e3

testdf2 = data_frame(
a = rnorm(n2),
b = rnorm(n2, mean = -1)
)

psych::ICC(testdf2 %>% t())
Call: psych::ICC(x = testdf2 %>% t())

Intraclass correlation coefficients
type  ICC    F df1   df2 p lower bound upper bound
Single_raters_absolute   ICC1 0.33 4946   1 19998 0        0.09           1
Single_random_raters     ICC2 0.33 4970   1  9999 0        0.09           1
Single_fixed_raters      ICC3 0.33 4970   1  9999 0        0.09           1
Average_raters_absolute ICC1k 1.00 4946   1 19998 0        1.00           1
Average_random_raters   ICC2k 1.00 4970   1  9999 0        1.00           1
Average_fixed_raters    ICC3k 1.00 4970   1  9999 0        1.00           1

Number of subjects = 2     Number of Judges =  10000


So far, so good. Now, imagine there is some measurement error in the estimates above. In fact, suppose we add some random measurement error corresponding to reliabilities from .9 to .1. Then we get this:

#reduce reliability while preserving distribution
reduce_reliability = function(x, reliability) {
x * sqrt(reliability) + rnorm(length(x), mean = 0, sd = (sqrt(1 - reliability)))
}

#loop and get values
rels = seq(.9, .1, by = -.1)
names(rels) = rels
ICC_noise = map_dbl(rels, function(r) {
#copy original
tmp = testdf2

tmp$$a = tmp$$a %>% reduce_reliability(reliability = r)
tmp$$b = tmp$$b %>% reduce_reliability(reliability = r)

#ICC
psych::ICC(tmp %>% t()) %>% .\$results %>% .[[1, 2]]
})
ICC_noise

0.9    0.8    0.7    0.6    0.5    0.4    0.3    0.2    0.1
0.3048 0.2845 0.2602 0.2187 0.2047 0.1688 0.1357 0.0902 0.0420


So, we see an expected decline in the values with increasing measurement error, but how does one adjust this back so we always get .33 here? Dividing by sqrt(reliability) as with the Pearson correlation gives incorrect results:

ICC_noise / sqrt(rels)
0.9   0.8   0.7   0.6   0.5   0.4   0.3   0.2   0.1
0.322 0.316 0.317 0.292 0.279 0.271 0.242 0.211 0.166

ICC_noise / rels
0.9   0.8   0.7   0.6   0.5   0.4   0.3   0.2   0.1
0.339 0.353 0.379 0.377 0.395 0.428 0.441 0.472 0.525


So, appears one has to use a different formula. However, I have been unable to find one.