# Questions tagged [intraclass-correlation]

Intraclass correlation coefficient (ICC) measures how much quantitative values are similar or associated within groups. There exist several versions and alternative formulations of ICC. The correlation is used to assess a dataset clusteredness, agreement among raters and in other settings.

290 questions
Filter by
Sorted by
Tagged with
12 views

### Design Effect in Cluster Randomised Trials

I want to understand why and how variance inflation factor is taken in account with the formula [1+(m-1)*ICC], where M= Size of clusters and ICC is the intracluster correlation. As I have deduced till ...
1 vote
29 views

### Negative ICC values for inter-rater reliability

I have conducted research which assessed the technical skills of surgeons. A surgeon will perform a simulated operation, and be assessed by two different assessors whilst completing the procedure. The ...
6 views

### Singluarity Issue when Computing ICCs for Unordered Pairs

I have 2 datasets of sibling pairs: one from divorced parents and one from married parents). Each pair has a unique "family" value. For this research question, it does not matter what order ...
28 views

### Do you need control variables in multi level regressions?

I am working with multilevel regressions. Do these need control variables? I ask as I am wondering if the independent variables grouping of multi level regressions is enough? In this case, the groups ...
1 vote
31 views

### multi level model and intraclass correlation coefficient

Let's say I have a data set with 4 variables: x1, x2, x3 and y. I have multiple measurements for every value of x1 and I want to perform linear regression. The goal of my research is to show proof ...
1 vote
41 views

### Understanding ICC and R2 in a MLM

I have a simple repeated measures mixed-level model with random intercept by participant. I am trying to understand the resulting R2 and ICC values. I have read numerous discussions on here, but none ...
1 vote
23 views

### Optimal Intraclass correlation coefficient computation method

I am hoping to compute intraclass correlations to evaluate interrater reliability (IRR) for interviews conduct with participants in a clinical psychology study. Specifically, we want to determine IRR ...
15 views

### Can you use inter-rater reliability to measure reliability of a whole measure with items measuring different things? (Krippendorff's Alpha or similar)

Let's say that you have one case study, where 30 raters are asked to apply a measure that has 5 items. Can you compute a single Krippendorff's Alpha (kalpha) value across all 5 items (that are the ...
18 views

### pbgtest, pwartest and pbltest yields different results

I am doing a econometrics assignments that asks me to evaluate the best panel data models for a production function. I have the following R code: ...
18 views

### Comparing ICCs between logistic and linear mixed-effects models

I have a (Bayesian) logistic mixed-effects model and a linear one, both fitted with brms. Both are fitted on data from human subjects that account for repetition, therefore, I have included a subject ...
39 views

### T test when there are different raters? I am not sure if my procedure is correct

I have two sets of translated text. The first set contains poor translations, and the second set includes perfect translations. I asked three evaluators to rate each of the translations. My objective ...
13 views

### help with choosing statistical test to compare responses to a questionnaire

I have a questionnaire made of 10 questions given to doctors from different specialties. Each doctor evaluates 10 echocardiographic images and gives the image a score of 1-10 based on their own ...
36 views

### Some Question about the Intraclass Correlation in interrater reliability

I am currently working on intraclass correlation in the context of inter-rater reliability. There are different forms of ICC (Shrout and Fleiss, 1979; McGraw and Wong, 1999) and according to the ...
15 views

### Agreement statistic for likert scale evaluation of different image modalities

I have some data of dental images. The same image was taken using the gold standard method and using an alternative new more cost effective method. Evaluators rate each image (blinded) based on a ...
39 views

### Disagreement between ICC and Wilcoxon Paired samples t-test

This is a short question. For the test-retest reliability I used both ICC and Wilcoxon Paired Samples t-test as measures. My data isn't normally distributed, so that's why I opted for Wilcoxon. ...
34 views

### How to extract pre/post correlations at each level of a multilevel model? (Cluster RCT - two levels - children clustered within schools)

I am analysing a randomised controlled trial using a two level linear mixed model. This is a two arm, pre-post design, where the post test is the primary outcome and we control for pre-test as a ...
36 views

### Finding the partial intraclass correlation

How do I calculate the partial correlation with an intraclass correlation? I tried asking earlier, but provided too much detail because it’s not easy to explain what I mean in a short or simple way. I’...
48 views

### Confused about partial intraclass correlation

Suppose the correlation between baldness scores in group 1 (fathers) and group 2 (sons) is 0.5. Furthermore, let the correlation between DHT and baldness scores in group 1 (fathers) be 0.4, and the ...
51 views

### Intra-rater reliability of an instrument

I'm working on the implementation of a "new version" of an existing instrument in the posture analysis. I know that if I want to compare the measures between the old one and the new one I ...
90 views

### Significant ICC value, but design effect <2: Is a mixed model approach justified?

An epidemiology study used mixed-level modelling to investigate the effect of individual and household-level exposures on malaria infection (Bannister-Tyrrell M. et al. Importance of household-level ...
29 views

### ICC and random slopes in Linear Mixed Effect Models

I'm currently creating a model with lme4 where I have the current formula: y ~ a + b + c + (1 + b + c|group) so that I get a random slope of b and c for each group....
1 vote
84 views

### What value of ICC to use when calculating effective sample size?

What value of ICC should I use when calculating the effective sample size of clustered data? A previous publication conducting a similar study reported ICCs of 0.04 for the full mixed model (random ...
20 views

### What intraclass correlation setup to use when there are a variable $k$ number of raters?

I have 200 subjects that need rating, each of which are given to between 3 to 8 raters (a rater may be able to grade more than one subject). I would like to compute the Intraclass Correlation but do ...
1 vote
30 views

### What statistical test would be appropriate for my study?

I am looking at the difference in flow rate on composition of coarse matter. I have set flow rate into two categories. Above and below 3700 L/s. I have raw data that consists (grams of matter for ...
50 views

### ICC and random effect - how to get ICC for each level of the group?

I am struggling a bit to connect the dots between intraclass correlation and random effects: Suppose that I have data from two groups - twins and non-twins (siblings). I am interested in calculating ...
11 views

### Is it possible to measure ICC for a group of six variables?

I have a group of 120 people. They all have a special bone with 6 variables that have been measured by 3 observers. I want to compare the means of each variable in one observer group with other ...
13 views

### Who are the raters and which ICC is valid for a 2-arm longitudinal RCT?

I'm totally confused by the terminology, as I never used it. There is a 2-arm randomized study with multiple timepoints: baseline (pre-treatment) and multiple post-treatment assessments. I need to ...
41 views

### Should we test only 20% of the sample for intraclass correlation coefficient?

I'm currently calculating an intraclass correlation coefficient for my data. As far as I'm concerned, the best choice (following this approach and the approach kindly recommended on the previous post ...
127 views

### inter-rater reliability for numeric variables

I've seen some questions on inter-rater reliability here for categorical variables. I'm wondering what would be an adequate approach to test inter-rater reliability between two raters for a numeric ...
53 views

### Is it okay to calculate intra-cluster correlation with unequal cluster sizes?

I have to calculate intra-cluster correlation of a score variable, but my cluster sizes are unequal i.e. the smallest has two observations and the largest has 32 observations. Is it okay to calculate ...
150 views

### Can I check the reliability of my test after collecting my data?

I have performed a study where I tested 2 groups, before and after two different types of manipulations. The main outcome of my study were the group differences at time point 2. Now someone asks what ...
93 views

### Variance of average of 𝑛 correlated random variable where 𝑛 is random variable also

I have a sum of n correlated variables $\sum_{i=1}^n X_i$ and would like to estimate the variance of the sum. I do it with two different methods. Firstly, I can decompose sum as multiplication of the ...
174 views

### Using ICC to compare repeated measurements from multiple raters

Statistics isn't really my cup of tea and I am struggling quite a bit with this. As a validation of a measuring method, I want to use the inter- and intra-rater reliability as a measure of 'goodness'. ...
70 views

### Group sequential design, repeatability/ reliability/ intra-class correlation coefficient (ICC) and sample size calculation

I want to test a survey taken by two nurses on the number of patients and determine the intra-class correlation coefficient/reliability. I want to determine the sample size for the analysis which will ...
93 views

### How is the "intraclass-correlation" related to the covariance matrices of the random-effects and residuals in a mixed-effects model?

How is the intraclass-correlation related to the covariance matrix of the random-effects and residuals? Beginning with Ronald Fisher, the intraclass correlation has been regarded within the framework ... 63 views

### Do we need to normalized data for computing intraclass correlation coefficient?

I want to calculate the ICC for data that have different scales. When I calculate ICC 1 way method with and without normalization data, the result is totally different. Do I need to normalize the data ...
1 vote
67 views

291 views

### Cronbach’s alpha not equal to ICC 3,k

I am calculating ICC over 16 items rated by 41 raters, using the ICC function from the psych package. The output is: ...
30 views

### Kruskal-Wallis test on data-augmented sample

summary of problem: I have a group of 15 images (15 samples). I have performed data augmentation on these (rotation, inserted noise, etc.): 7 variations of each image (+original img), resulting in a ...
383 views

### calculating ICC manually versus the performance package

I reproduced an example I found on Stackexchange, here, and I failed to get the same ICC using the performance::icc package in R. My reproducible example is below. Can someone explain to me why the ...
233 views

### Why is the ICC near 0 in this longitudinal data?

I’m working with a toy data set to get a basic understanding of longitudinal data analysis and I’m having a difficult time with a conceptual understanding of the intraclass correlation (ICC) as a ...
51 views

### Intra-class correlation coefficients with R

I have recorded the heart rate of a number of participants with two methods: ECG rPPG I am trying to demonstrate that the rPPG is a reliable measure and I think I can use an approach similar to this ... 1 vote
153 views

### Confidence Intervals for ICC

I was wondering if anyone might know of a way to calculate confidence intervals around an ICC(1) value? I'm running a multilevel model using the lmer() function in lme4 where I'm interested in seeing ...
210 views

### P-values for ICC(1) in R

I am currently running multilevel analyses in R to analyze data for a daily diary study where observations are nested within people. I am interested in identifying how much of the total variance in my ...
23 views

### Consistency across Data for group level

I want to design a MLM model that shows there is inconsistency in Y variable on group level. Any ideas on how to do that? When I add more predictors, the ICC goes up. But I need to demonstrate that ...
1 vote
93 views

### Can you average ICC values for Intra-rater reliability?

I am using ICCs to test the inter- and intra-rater reliability for a body condition scoring system. I have two different groups of evaluators: specialists and students. I would like to evaluate the ...