Skip to main content

Questions tagged [intraclass-correlation]

Intraclass correlation coefficient (ICC) measures how much quantitative values are similar or associated within groups. There exist several versions and alternative formulations of ICC. The correlation is used to assess a dataset clusteredness, agreement among raters and in other settings.

Filter by
Sorted by
Tagged with
2 votes
1 answer
76 views

Heterogeneity and ICC when using metafor's rma.mv and robust variance estimation

I am conducting a meta-analysis of standardised mean differences with many sources of dependency (multiple outcomes measuring the same construct, multiple control groups, outcomes from dyads (e.g., ...
AndreasV's user avatar
1 vote
0 answers
14 views

Why does the ICC differ when comparing multilevel models (log transformations vs regular scale) of the outcome variable in R using lme4?

I'm currently working on a multilevel modeling project in R utilizing the lme4 package. The primary aim of the research is to assess the relative importance of between-family and within-family ...
Max Herre's user avatar
0 votes
0 answers
12 views

Why does the ICC differ when comparing multilevel models with using log transformations of the outcome variable in R using lme4?

I'm currently working on a multilevel modeling project in R utilizing the lme4 package. The primary aim of the research is to assess the relative importance of between-family and within-family ...
Max Herre's user avatar
0 votes
0 answers
19 views

How to format data for one-way intraclass correlation coefficient?

I'm testing inter-rater reliability of an instrument on a sample of subjects from several sites. Each subject is assessed by two raters, different between each site. According to this paper, I should ...
Charlie's user avatar
  • 142
0 votes
0 answers
5 views

How to Calculate Number of Repeated Tests to Achieve a Target ICC?

I am designing a case study and will be measuring something at baseline, periodically over time, and at post-testing. I will use the baseline measures to be able to calculate the 'minimum detectable ...
Jake Remmert's user avatar
0 votes
0 answers
11 views

How to measure the difference in outcomes across 32 dimensions in a time-series study?

I have a dataset that recorded the contact patterns for each participant during three waves. For example: matrix_base_1 means that Participant 1 met with one person aged over 70 at location two, and ...
Chao's user avatar
  • 1
2 votes
0 answers
28 views

Is the Between-Groups Variance a Covariance?

I am currently working through a book/class in quantitative genetics, and in Falconer and Mackay's Introduction to Quantitative Genetics, the following line stumped me: "The between-group ...
The_Scientist___'s user avatar
2 votes
0 answers
34 views

Is there a statistic that expresses whether two measures tell us the same information?

After trying to understand different implementations of intraclass correlation (ICC), inter-rater reliability, and bland-altman statistics, I'm not quite sure what is the right choice here, and would ...
stck8888's user avatar
0 votes
1 answer
145 views

Intraclass correlation - CI 95% in negative values?

I have my data + data from colleagues who measured independently. I have to create an ICC between my results and theirs. But the only problem is that even if I got some ICC results plus 95% CI, I don'...
Diana's user avatar
  • 1
0 votes
0 answers
19 views

Design Effect in Cluster Randomised Trials

I want to understand why and how variance inflation factor is taken in account with the formula [1+(m-1)*ICC], where M= Size of clusters and ICC is the intracluster correlation. As I have deduced till ...
Angela Sarkar's user avatar
1 vote
0 answers
47 views

Negative ICC values for inter-rater reliability

I have conducted research which assessed the technical skills of surgeons. A surgeon will perform a simulated operation, and be assessed by two different assessors whilst completing the procedure. The ...
DLCN1994's user avatar
0 votes
0 answers
7 views

Singluarity Issue when Computing ICCs for Unordered Pairs

I have 2 datasets of sibling pairs: one from divorced parents and one from married parents). Each pair has a unique "family" value. For this research question, it does not matter what order ...
df_postgrad's user avatar
0 votes
1 answer
46 views

Do you need control variables in multi level regressions?

I am working with multilevel regressions. Do these need control variables? I ask as I am wondering if the independent variables grouping of multi level regressions is enough? In this case, the groups ...
MLux's user avatar
  • 28
1 vote
1 answer
181 views

multi level model and intraclass correlation coefficient

Let's say I have a data set with 4 variables: x1, x2, x3 and y. I have multiple measurements for every value of x1 and I want to perform linear regression. The goal of my research is to show proof ...
Melanie's user avatar
  • 133
1 vote
0 answers
88 views

Understanding ICC and R2 in a MLM

I have a simple repeated measures mixed-level model with random intercept by participant. I am trying to understand the resulting $R^2$ and ICC values. I have read numerous discussions on here, but ...
Trypanosoma's user avatar
1 vote
0 answers
29 views

Optimal Intraclass correlation coefficient computation method

I am hoping to compute intraclass correlations to evaluate interrater reliability (IRR) for interviews conduct with participants in a clinical psychology study. Specifically, we want to determine IRR ...
Wyoming Psychology's user avatar
0 votes
0 answers
20 views

Can you use inter-rater reliability to measure reliability of a whole measure with items measuring different things? (Krippendorff's Alpha or similar)

Let's say that you have one case study, where 30 raters are asked to apply a measure that has 5 items. Can you compute a single Krippendorff's Alpha (kalpha) value across all 5 items (that are the ...
jstat's user avatar
  • 1
0 votes
0 answers
46 views

pbgtest, pwartest and pbltest yields different results

I am doing a econometrics assignments that asks me to evaluate the best panel data models for a production function. I have the following R code: ...
VittorioRossi02's user avatar
0 votes
0 answers
25 views

Comparing ICCs between logistic and linear mixed-effects models

I have a (Bayesian) logistic mixed-effects model and a linear one, both fitted with brms. Both are fitted on data from human subjects that account for repetition, therefore, I have included a subject ...
Tester01's user avatar
2 votes
1 answer
42 views

T test when there are different raters? I am not sure if my procedure is correct

I have two sets of translated text. The first set contains poor translations, and the second set includes perfect translations. I asked three evaluators to rate each of the translations. My objective ...
chen Crush's user avatar
2 votes
0 answers
63 views

Some Question about the Intraclass Correlation in interrater reliability

I am currently working on intraclass correlation in the context of inter-rater reliability. There are different forms of ICC (Shrout and Fleiss, 1979; McGraw and Wong, 1999) and according to the ...
バシル's user avatar
  • 123
0 votes
1 answer
58 views

Finding the partial intraclass correlation

How do I calculate the partial correlation with an intraclass correlation? I tried asking earlier, but provided too much detail because it’s not easy to explain what I mean in a short or simple way. I’...
Testosterone999's user avatar
0 votes
0 answers
54 views

Confused about partial intraclass correlation

Suppose the correlation between baldness scores in group 1 (fathers) and group 2 (sons) is 0.5. Furthermore, let the correlation between DHT and baldness scores in group 1 (fathers) be 0.4, and the ...
Testosterone999's user avatar
0 votes
0 answers
77 views

Intra-rater reliability of an instrument

I'm working on the implementation of a "new version" of an existing instrument in the posture analysis. I know that if I want to compare the measures between the old one and the new one I ...
Fede R's user avatar
  • 23
3 votes
1 answer
127 views

Significant ICC value, but design effect <2: Is a mixed model approach justified?

An epidemiology study used mixed-level modelling to investigate the effect of individual and household-level exposures on malaria infection (Bannister-Tyrrell M. et al. Importance of household-level ...
Trypanosoma's user avatar
1 vote
0 answers
110 views

What value of ICC to use when calculating effective sample size?

What value of ICC should I use when calculating the effective sample size of clustered data? A previous publication conducting a similar study reported ICCs of 0.04 for the full mixed model (random ...
Trypanosoma's user avatar
1 vote
1 answer
32 views

What statistical test would be appropriate for my study?

I am looking at the difference in flow rate on composition of coarse matter. I have set flow rate into two categories. Above and below 3700 L/s. I have raw data that consists (grams of matter for ...
Walt's user avatar
  • 11
1 vote
1 answer
104 views

Should we test only 20% of the sample for intraclass correlation coefficient?

I'm currently calculating an intraclass correlation coefficient for my data. As far as I'm concerned, the best choice (following this approach and the approach kindly recommended on the previous post ...
Larissa Cury's user avatar
2 votes
1 answer
366 views

inter-rater reliability for numeric variables

I've seen some questions on inter-rater reliability here for categorical variables. I'm wondering what would be an adequate approach to test inter-rater reliability between two raters for a numeric ...
Larissa Cury's user avatar
3 votes
1 answer
249 views

Can I check the reliability of my test after collecting my data?

I have performed a study where I tested 2 groups, before and after two different types of manipulations. The main outcome of my study were the group differences at time point 2. Now someone asks what ...
Inkling's user avatar
  • 299
0 votes
1 answer
146 views

Variance of average of 𝑛 correlated random variable where 𝑛 is random variable also

I have a sum of n correlated variables $\sum_{i=1}^n X_i$ and would like to estimate the variance of the sum. I do it with two different methods. Firstly, I can decompose sum as multiplication of the ...
koch_kir's user avatar
0 votes
0 answers
106 views

How is the "intraclass-correlation" related to the covariance matrices of the random-effects and residuals in a mixed-effects model?

How is the intraclass-correlation related to the covariance matrix of the random-effects and residuals? Beginning with Ronald Fisher, the intraclass correlation has been regarded within the framework ...
user avatar
0 votes
0 answers
107 views

Do we need to normalized data for computing intraclass correlation coefficient?

I want to calculate the ICC for data that have different scales. When I calculate ICC 1 way method with and without normalization data, the result is totally different. Do I need to normalize the data ...
Kate's user avatar
  • 1
1 vote
1 answer
74 views

Failing to recover correlation parameter from a linear fixed effects model with correlated errors simulation

I am trying to simulate some data from a linear fixed effects model with correlated errors. The model is quite simple, inspired in a pre-post study. $$Y_i|X_i=x_i\sim N(x_i\beta,\Sigma_i)$$ where $Y_i=...
Nicolas Molano's user avatar
1 vote
0 answers
48 views

Effects of Subject variance and scale in ICC (intraclass correlation coefficient)

I am computing Two way fixed effects, absolute agreement, multiple measurements ICC on two different measurements. I have 3 measurements of the same subject from my instrument. The measurements are ...
TheNumber23's user avatar
1 vote
1 answer
231 views

What is the right statistical measure for consistency?

I have ordinal data from a Likert survey with $45$ questions and $15$ participants. Every participant took the survey six times, so I have six answers from every participant for every question. All ...
Karla's user avatar
  • 143
2 votes
2 answers
378 views

Cronbach’s alpha not equal to ICC 3,k

I am calculating ICC over 16 items rated by 41 raters, using the ICC function from the psych package. The output is: ...
Alessio's user avatar
  • 21
0 votes
0 answers
37 views

Kruskal-Wallis test on data-augmented sample

summary of problem: I have a group of 15 images (15 samples). I have performed data augmentation on these (rotation, inserted noise, etc.): 7 variations of each image (+original img), resulting in a ...
Kate S.D.'s user avatar
0 votes
2 answers
641 views

calculating ICC manually versus the performance package

I reproduced an example I found on Stackexchange, here, and I failed to get the same ICC using the performance::icc package in R. My reproducible example is below. Can someone explain to me why the ...
AJ_0000's user avatar
3 votes
1 answer
328 views

Why is the ICC near 0 in this longitudinal data?

I’m working with a toy data set to get a basic understanding of longitudinal data analysis and I’m having a difficult time with a conceptual understanding of the intraclass correlation (ICC) as a ...
LISpace's user avatar
  • 33
2 votes
0 answers
56 views

Intra-class correlation coefficients with R

I have recorded the heart rate of a number of participants with two methods: ECG rPPG I am trying to demonstrate that the rPPG is a reliable measure and I think I can use an approach similar to this ...
user avatar
1 vote
0 answers
194 views

Confidence Intervals for ICC

I was wondering if anyone might know of a way to calculate confidence intervals around an ICC(1) value? I'm running a multilevel model using the lmer() function in lme4 where I'm interested in seeing ...
user360342's user avatar
0 votes
0 answers
283 views

P-values for ICC(1) in R

I am currently running multilevel analyses in R to analyze data for a daily diary study where observations are nested within people. I am interested in identifying how much of the total variance in my ...
ccoul's user avatar
  • 31
0 votes
0 answers
23 views

Consistency across Data for group level

I want to design a MLM model that shows there is inconsistency in Y variable on group level. Any ideas on how to do that? When I add more predictors, the ICC goes up. But I need to demonstrate that ...
Kawi's user avatar
  • 11
1 vote
0 answers
134 views

Can you average ICC values for Intra-rater reliability?

I am using ICCs to test the inter- and intra-rater reliability for a body condition scoring system. I have two different groups of evaluators: specialists and students. I would like to evaluate the ...
Nelsok's user avatar
  • 11
0 votes
0 answers
40 views

appropriate ICC

My apologies this is a rookie question, I am aware of what ICC(Intraclass correlation) is, and that it measures how strongly subjects in the same group resemble each other. However I am not sure how ...
Ahir Bhairav Orai's user avatar
0 votes
0 answers
663 views

Computing ICCs in Matlab, to assess rater consistency (inter-rater agreement)

In an experiment, participants rated each of several musical stimuli on 5 aspects (ratings A to E), across ~3,000 trials. Stimuli were performed (by a previous set of subjects) either solo or duo (...
z8080's user avatar
  • 2,382
1 vote
0 answers
92 views

Which intraclass correlation coefficient (ICC) to use to verify the test-retest reliability of a questionnaire?

I want to check the test-retest reliability of a questionnaire. -A total of 300 adolescents from the same school (10 classes) participated in the study. -The instrument was applied in two moments with ...
dr_stats's user avatar
1 vote
0 answers
49 views

ICC in Random intercept model with a predictor variable [closed]

Are we able/ or is it correct to calculate ICC in random intercept model which has a preditor variable within? Does this value give us the variation of the outcome which is purely related to subject ...
loreen's user avatar
  • 11
1 vote
2 answers
362 views

Calculate intra-class correlation (or some other measure of reliability) with variable number of raters

I have some stimuli rated by users. I want to estimate how much of the variance in ratings is due to the stimuli, so that I can compare between various types of ratings which are the most reliable. ...
user2288107's user avatar

1
2 3 4 5 6