Questions tagged [intraclass-correlation]
Intraclass correlation coefficient (ICC) measures how much quantitative values are similar or associated within groups. There exist several versions and alternative formulations of ICC. The correlation is used to assess a dataset clusteredness, agreement among raters and in other settings.
290
questions
0
votes
0
answers
12
views
Design Effect in Cluster Randomised Trials
I want to understand why and how variance inflation factor is taken in account with the formula [1+(m-1)*ICC], where M= Size of clusters and ICC is the intracluster correlation.
As I have deduced till ...
1
vote
0
answers
29
views
Negative ICC values for inter-rater reliability
I have conducted research which assessed the technical skills of surgeons.
A surgeon will perform a simulated operation, and be assessed by two different assessors whilst completing the procedure. The ...
- 11
0
votes
0
answers
6
views
Singluarity Issue when Computing ICCs for Unordered Pairs
I have 2 datasets of sibling pairs: one from divorced parents and one from married parents). Each pair has a unique "family" value. For this research question, it does not matter what order ...
0
votes
1
answer
28
views
Do you need control variables in multi level regressions?
I am working with multilevel regressions. Do these need control variables? I ask as I am wondering if the independent variables grouping of multi level regressions is enough? In this case, the groups ...
- 28
1
vote
1
answer
31
views
multi level model and intraclass correlation coefficient
Let's say I have a data set with 4 variables: x1, x2, x3 and y. I have multiple measurements for every value of x1 and I want to perform linear regression. The goal of my research is to show proof ...
- 113
1
vote
0
answers
41
views
Understanding ICC and R2 in a MLM
I have a simple repeated measures mixed-level model with random intercept by participant. I am trying to understand the resulting R2 and ICC values. I have read numerous discussions on here, but none ...
- 97
1
vote
0
answers
23
views
Optimal Intraclass correlation coefficient computation method
I am hoping to compute intraclass correlations to evaluate interrater reliability (IRR) for interviews conduct with participants in a clinical psychology study. Specifically, we want to determine IRR ...
0
votes
0
answers
15
views
Can you use inter-rater reliability to measure reliability of a whole measure with items measuring different things? (Krippendorff's Alpha or similar)
Let's say that you have one case study, where 30 raters are asked to apply a measure that has 5 items. Can you compute a single Krippendorff's Alpha (kalpha) value across all 5 items (that are the ...
- 1
0
votes
0
answers
18
views
pbgtest, pwartest and pbltest yields different results
I am doing a econometrics assignments that asks me to evaluate the best panel data models for a production function.
I have the following R code:
...
0
votes
0
answers
18
views
Comparing ICCs between logistic and linear mixed-effects models
I have a (Bayesian) logistic mixed-effects model and a linear one, both fitted with brms. Both are fitted on data from human subjects that account for repetition, therefore, I have included a subject ...
- 45
2
votes
1
answer
39
views
T test when there are different raters? I am not sure if my procedure is correct
I have two sets of translated text. The first set contains poor translations, and the second set includes perfect translations. I asked three evaluators to rate each of the translations. My objective ...
- 21
0
votes
0
answers
13
views
help with choosing statistical test to compare responses to a questionnaire
I have a questionnaire made of 10 questions given to doctors from different specialties.
Each doctor evaluates 10 echocardiographic images and gives the image a score of 1-10 based on their own ...
2
votes
0
answers
36
views
Some Question about the Intraclass Correlation in interrater reliability
I am currently working on intraclass correlation in the context of inter-rater reliability. There are different forms of ICC (Shrout and Fleiss, 1979; McGraw and Wong, 1999) and according to the ...
- 93
0
votes
0
answers
15
views
Agreement statistic for likert scale evaluation of different image modalities
I have some data of dental images. The same image was taken using the gold standard method and using an alternative new more cost effective method. Evaluators rate each image (blinded) based on a ...
- 21
0
votes
0
answers
39
views
Disagreement between ICC and Wilcoxon Paired samples t-test
This is a short question.
For the test-retest reliability I used both ICC and Wilcoxon Paired Samples t-test as measures. My data isn't normally distributed, so that's why I opted for Wilcoxon.
...
- 1
0
votes
0
answers
34
views
How to extract pre/post correlations at each level of a multilevel model? (Cluster RCT - two levels - children clustered within schools)
I am analysing a randomised controlled trial using a two level linear mixed model. This is a two arm, pre-post design, where the post test is the primary outcome and we control for pre-test as a ...
- 333
0
votes
1
answer
36
views
Finding the partial intraclass correlation
How do I calculate the partial correlation with an intraclass correlation? I tried asking earlier, but provided too much detail because it’s not easy to explain what I mean in a short or simple way. I’...
0
votes
0
answers
48
views
Confused about partial intraclass correlation
Suppose the correlation between baldness scores in group 1 (fathers) and group 2 (sons) is 0.5. Furthermore, let the correlation between DHT and baldness scores in group 1 (fathers) be 0.4, and the ...
0
votes
0
answers
51
views
Intra-rater reliability of an instrument
I'm working on the implementation of a "new version" of an existing instrument in the posture analysis.
I know that if I want to compare the measures between the old one and the new one I ...
- 23
2
votes
1
answer
90
views
Significant ICC value, but design effect <2: Is a mixed model approach justified?
An epidemiology study used mixed-level modelling to investigate the effect of individual and household-level exposures on malaria infection (Bannister-Tyrrell M. et al. Importance of household-level ...
- 97
0
votes
0
answers
29
views
ICC and random slopes in Linear Mixed Effect Models
I'm currently creating a model with lme4 where I have the current formula:
y ~ a + b + c + (1 + b + c|group)
so that I get a random slope of b and c for each group....
- 1
1
vote
0
answers
84
views
What value of ICC to use when calculating effective sample size?
What value of ICC should I use when calculating the effective sample size of clustered data?
A previous publication conducting a similar study reported ICCs of 0.04 for the full mixed model (random ...
- 97
0
votes
0
answers
20
views
What intraclass correlation setup to use when there are a variable $k$ number of raters?
I have 200 subjects that need rating, each of which are given to between 3 to 8 raters (a rater may be able to grade more than one subject). I would like to compute the Intraclass Correlation but do ...
- 3,452
1
vote
1
answer
30
views
What statistical test would be appropriate for my study?
I am looking at the difference in flow rate on composition of coarse matter.
I have set flow rate into two categories. Above and below 3700 L/s.
I have raw data that consists (grams of matter for ...
- 11
0
votes
0
answers
50
views
ICC and random effect - how to get ICC for each level of the group?
I am struggling a bit to connect the dots between intraclass correlation and random effects:
Suppose that I have data from two groups - twins and non-twins (siblings). I am interested in calculating ...
- 215
0
votes
0
answers
11
views
Is it possible to measure ICC for a group of six variables?
I have a group of 120 people. They all have a special bone with 6 variables that have been measured by 3 observers. I want to compare the means of each variable in one observer group with other ...
0
votes
0
answers
13
views
Who are the raters and which ICC is valid for a 2-arm longitudinal RCT?
I'm totally confused by the terminology, as I never used it.
There is a 2-arm randomized study with multiple timepoints: baseline (pre-treatment) and multiple post-treatment assessments.
I need to ...
- 1
0
votes
1
answer
41
views
Should we test only 20% of the sample for intraclass correlation coefficient?
I'm currently calculating an intraclass correlation coefficient for my data. As far as I'm concerned, the best choice (following this approach and the approach kindly recommended on the previous post ...
- 641
2
votes
1
answer
127
views
inter-rater reliability for numeric variables
I've seen some questions on inter-rater reliability here for categorical variables. I'm wondering what would be an adequate approach to test inter-rater reliability between two raters for a numeric ...
- 641
0
votes
0
answers
53
views
Is it okay to calculate intra-cluster correlation with unequal cluster sizes?
I have to calculate intra-cluster correlation of a score variable, but my cluster sizes are unequal i.e. the smallest has two observations and the largest has 32 observations. Is it okay to calculate ...
- 13
2
votes
1
answer
150
views
Can I check the reliability of my test after collecting my data?
I have performed a study where I tested 2 groups, before and after two different types of manipulations. The main outcome of my study were the group differences at time point 2.
Now someone asks what ...
- 289
0
votes
1
answer
93
views
Variance of average of 𝑛 correlated random variable where 𝑛 is random variable also
I have a sum of n correlated variables $\sum_{i=1}^n X_i$ and would like to estimate the variance of the sum. I do it with two different methods.
Firstly, I can decompose sum as multiplication of the ...
- 77
0
votes
0
answers
174
views
Using ICC to compare repeated measurements from multiple raters
Statistics isn't really my cup of tea and I am struggling quite a bit with this.
As a validation of a measuring method, I want to use the inter- and intra-rater reliability as a measure of 'goodness'. ...
0
votes
0
answers
70
views
Group sequential design, repeatability/ reliability/ intra-class correlation coefficient (ICC) and sample size calculation
I want to test a survey taken by two nurses on the number of patients and determine the intra-class correlation coefficient/reliability. I want to determine the sample size for the analysis which will ...
- 1
0
votes
0
answers
93
views
How is the "intraclass-correlation" related to the covariance matrices of the random-effects and residuals in a mixed-effects model?
How is the intraclass-correlation related to the covariance matrix of the random-effects and residuals?
Beginning with Ronald Fisher, the intraclass correlation has been regarded within the framework ...
user318514
0
votes
0
answers
63
views
Do we need to normalized data for computing intraclass correlation coefficient?
I want to calculate the ICC for data that have different scales. When I calculate ICC 1 way method with and without normalization data, the result is totally different. Do I need to normalize the data ...
- 1
1
vote
1
answer
67
views
Failing to recover correlation parameter from a linear fixed effects model with correlated errors simulation
I am trying to simulate some data from a linear fixed effects model with correlated errors. The model is quite simple, inspired in a pre-post study.
$$Y_i|X_i=x_i\sim N(x_i\beta,\Sigma_i)$$
where $Y_i=...
1
vote
0
answers
38
views
Effects of Subject variance and scale in ICC (intraclass correlation coefficient)
I am computing Two way fixed effects, absolute agreement, multiple measurements ICC on two different measurements. I have 3 measurements of the same subject from my instrument. The measurements are ...
- 121
1
vote
1
answer
149
views
What is the right statistical measure for consistency?
I have ordinal data from a Likert survey with $45$ questions and $15$ participants. Every participant took the survey six times, so I have six answers from every participant for every question. All ...
- 143
2
votes
2
answers
291
views
Cronbach’s alpha not equal to ICC 3,k
I am calculating ICC over 16 items rated by 41 raters, using the ICC function from the psych package. The output is:
...
- 21
0
votes
0
answers
30
views
Kruskal-Wallis test on data-augmented sample
summary of problem:
I have a group of 15 images (15 samples). I have performed data augmentation on these (rotation, inserted noise, etc.): 7 variations of each image (+original img), resulting in a ...
0
votes
2
answers
383
views
calculating ICC manually versus the performance package
I reproduced an example I found on Stackexchange, here, and I failed to get the same ICC using the performance::icc package in R. My reproducible example is below. Can someone explain to me why the ...
- 1
3
votes
1
answer
233
views
Why is the ICC near 0 in this longitudinal data?
I’m working with a toy data set to get a basic understanding of longitudinal data analysis and I’m having a difficult time with a conceptual understanding of the intraclass correlation (ICC) as a ...
- 33
2
votes
0
answers
51
views
Intra-class correlation coefficients with R
I have recorded the heart rate of a number of participants with two methods:
ECG
rPPG
I am trying to demonstrate that the rPPG is a reliable measure and I think I can use an approach similar to this ...
Glu
1
vote
0
answers
153
views
Confidence Intervals for ICC
I was wondering if anyone might know of a way to calculate confidence intervals around an ICC(1) value? I'm running a multilevel model using the lmer() function in lme4 where I'm interested in seeing ...
- 71
0
votes
0
answers
210
views
P-values for ICC(1) in R
I am currently running multilevel analyses in R to analyze data for a daily diary study where observations are nested within people. I am interested in identifying how much of the total variance in my ...
- 21
0
votes
0
answers
23
views
Consistency across Data for group level
I want to design a MLM model that shows there is inconsistency in Y variable on group level. Any ideas on how to do that? When I add more predictors, the ICC goes up. But I need to demonstrate that ...
- 11
1
vote
0
answers
93
views
Can you average ICC values for Intra-rater reliability?
I am using ICCs to test the inter- and intra-rater reliability for a body condition scoring system. I have two different groups of evaluators: specialists and students. I would like to evaluate the ...
- 11
0
votes
0
answers
38
views
appropriate ICC
My apologies this is a rookie question, I am aware of what ICC(Intraclass correlation) is, and that it measures how strongly subjects in the same group resemble each other. However I am not sure how ...
0
votes
0
answers
503
views
Computing ICCs in Matlab, to assess rater consistency (inter-rater agreement)
In an experiment, participants rated each of several musical stimuli on 5 aspects (ratings A to E), across ~3,000 trials. Stimuli were performed (by a previous set of subjects) either solo or duo (...
- 2,120