Questions tagged [intraclass-correlation]

Intraclass correlation coefficient (ICC) measures how much quantitative values are similar or associated within groups. There exist several versions and alternative formulations of ICC. The correlation is used to assess a dataset clusteredness, agreement among raters and in other settings.

Filter by
Sorted by
Tagged with
0 votes
0 answers
23 views

Using and Reporting ICC for Aggregated Data About Groups of People

I've designed a survey that consists of multiple questionnaires with Likert-scale questions, each of which targets a specific variable in my research model. The participants of this survey are ...
user avatar
0 votes
0 answers
16 views

How to decompose variance into intra and inter group variance for random effect

I have difficulty understanding something that may be something simple. It concerns the decomposition of variance between inter and intragroup variance. Here is how I understand it so far: If I have a ...
user avatar
  • 211
0 votes
0 answers
15 views

In a 1-way ANOVA, is the square-root of the coefficient of determination the intra-class correlation?

The actual question I mean to ask is if the square-root of the effect size is the intra-class correlation (ICC). When the 1-way ANOVA test is viewed as a bivariate analysis between a categorical ...
user avatar
  • 3,863
0 votes
0 answers
23 views

appropriate ICC

My apologies this is a rookie question, I am aware of what ICC(Intraclass correlation) is, and that it measures how strongly subjects in the same group resemble each other. However I am not sure how ...
user avatar
0 votes
0 answers
12 views

The meaning of "ICC corrected by age and comorbidity is best described in detail"

I received feedback on a manuscript comparing the mortality of two populations. Both crude and adjusted mortality rates are compared. Adjusting was done for age and comorbidity score and it is clearly ...
user avatar
  • 1,533
0 votes
0 answers
74 views

Computing ICCs in Matlab, to assess rater consistency (inter-rater agreement)

In an experiment, participants rated each of several musical stimuli on 5 aspects (ratings A to E), across ~3,000 trials. Stimuli were performed (by a previous set of subjects) either solo or duo (...
user avatar
  • 1,548
1 vote
0 answers
15 views

Which intraclass correlation coefficient (ICC) to use to verify the test-retest reliability of a questionnaire?

I want to check the test-retest reliability of a questionnaire. -A total of 300 adolescents from the same school (10 classes) participated in the study. -The instrument was applied in two moments with ...
user avatar
1 vote
0 answers
12 views

ICC in Random intercept model with a predictor variable

Are we able/ or is it correct to calculate ICC in random intercept model which has a preditor variable within? Does this value give us the variation of the outcome which is purely related to subject ...
user avatar
  • 11
0 votes
0 answers
28 views

Calculate intra-class correlation (or some other measure of reliability) with variable number of raters

I have some stimuli rated by users. I want to estimate how much of the variance in ratings is due to the stimuli, so that I can compare between various types of ratings which are the most reliable. ...
user avatar
0 votes
0 answers
76 views

Is this documentation for the R 'psych' package ICC() function incorrect?

I am not a statistician, but from my reading about various types of intraclass correlation measures, ICC(2,1) (Shrout and Fleiss nomenclature) is a measure of absolute agreement sensitive to additive ...
user avatar
  • 145
0 votes
0 answers
17 views

Is there a formula expressing the relationship between standard deviation, standard deviation of cluster effect and standard deviation within cluster?

Suppose I have data on child test scores of children from different schools. Children are the unit of observation and the schools are clusters. The standard deviation of the test score variable is 1.5....
user avatar
  • 13
1 vote
0 answers
40 views

ICC vs Repeated Measures ANOVA to Justify Multilevel Model

I have heard of using ICCs to determine if there is variability to justify using a multilevel (i.e., mixed effects, hierarchical) model. However, I've recently seen studies examine whether there is a ...
user avatar
  • 21
0 votes
0 answers
20 views

Intraclass correlation coefficiant for underdispersed count data

I would like to calculate the intraclass correlation coefficiant (ICC). My data are counts, approximately normal distributed and the variance is smaller than the mean. If I am not mistaken, this ...
user avatar
  • 11
0 votes
0 answers
12 views

Test-retest reliability with varying number of measurements

I developed a new method to measure a certain quantity on a radiological image of a patient. I have available varying numbers of studies per patient. I applied the method to all images available for ...
user avatar
1 vote
0 answers
23 views

What is the best way to compare fixed and random effects of a GLMM?

I am doing a research in which i am trying to measure the importance of the doctor that is in charge of a patient in a medical decision. For that (and others reasons) i have used a GLMM using lmer4 ...
user avatar
0 votes
0 answers
14 views

Estimating confidence intervals for ICCs in a multilevel Poisson model

Can anyone help me with this in R? I need the conf. intervals for the ICC in a glmmTMB model. I know I should either go for bootstrapping or something Raykov 2015 ...
user avatar
1 vote
0 answers
19 views

How to conduct a meta-analysis on ICCs (i.e., test-retest reliability)?

I have calculated two-way single consistency ICCs in R using the irr package. There are multiple ICCs representing test-retest reliability across these time points. Is there a way to compute a meta-...
user avatar
  • 309
1 vote
0 answers
53 views

What is the correct nesting structure for calculating the Inter-rater reliability or intra class correlation (ICC) with multiple raters?

I need to figure out which inter-rater reliability for multiple raters I should calculate. I'm trying to figure out how I should nest or structure certain observations within my dataset in order to ...
user avatar
0 votes
0 answers
98 views

Alternative to Intra-class correlation coefficient (ICC) for continuous data

I am working on a research project that involves a reliability study. In this case, 3 raters have assigned two sets of scores to 50 items: an ordinal score on a scale from 1 - 3, and a continuous ...
user avatar
2 votes
1 answer
97 views

Difference between Multilevel I^2 and ICC?

I have been using the excellent metafor package for a work-related project and for some reason there's a concept that I can't wrap my head around. In his guide (...
user avatar
1 vote
1 answer
43 views

Weighted Kappa with rating variables that have different set of categories

I want to measure the inter-agreement rate between two physicians that rated the same set of images with two different methods (rater1 used method1, rater2 used method2) BUT Rater1 rated the the ...
user avatar
  • 457
0 votes
0 answers
22 views

Intraclass correlation with skewed dataset

There is data on 20 people taking the same test 5 times each to look at the repeatability of the test. I understand that intraclass correlation is usually the best test to use to look at a question ...
user avatar
0 votes
0 answers
14 views

crude and adjusted Intra Class Correlation (ICC)

I have a data set with four different hormones which were assessed at two time points on 40 healthy participants. For these hormones I want to analyse the intra-individual stability of plasma ...
user avatar
0 votes
0 answers
12 views

Assessing Consistency and Agreement of Two Groups of Raters in an ordinal dataset

I'm using a real dataset of convenience to look at one group of subjects who are being assessed for "frailty". The goal is to see whether assessments by non-experts are comparable to experts....
user avatar
  • 1
1 vote
1 answer
157 views

Disagreement between ICC and paired t-test

I am trying to determine whether the values obtained by an algorithm calculation are systematically different from the the criterion values. The criterion values are obtained via manual observation ...
user avatar
  • 43
9 votes
2 answers
411 views

How to estimate the variance of correlated observations?

Assume we have n observations $x_i$ (i from 1 to n), each from the a normal distribution with mean 0 and some variance component: $X_i \sim N(0, \sigma^2)$. The random variables $X_i$s have some (let'...
user avatar
0 votes
0 answers
46 views

How to compare the test-retest reliability of two groups?

I have one group (NBD, N=23) and another group (PWA, N=22) who each have data at Test (time point 1) and retest (time point 2). I've calculated test-retest reliability using a oneway, consistency, ...
user avatar
0 votes
0 answers
51 views

Inter and intra rater reliability (intra class correlation) from linear mixed model

This is a follow-up question to this one. My dataset consists of: one dependent variable (value) measured 3 times (observation) ...
user avatar
  • 105
2 votes
0 answers
325 views

How to calculate two-way mixed ICC?

I want to calculate ICC for two-way mixed model, single rater, and either for consistency and absolute agreement. I found two packages for calculating ICC (intra-class correlation coefficient), that ...
user avatar
1 vote
0 answers
34 views

Compare within- and between-individuals variances with lme4?

How can I compare the variances of two random effects variables? I have an experiment with several proteins measured in individuals (ind = 4 levels) before and after a treatment (treatment = 2 levels ...
user avatar
  • 115
4 votes
0 answers
81 views

Calculating ICC for a beta-binomial GLMM

I understand that ICC in binomial GLMMs with a logit link can be calculated via R, where the residual deviance is (pi ^ 2) / 3. However, this is assuming that the ...
user avatar
  • 41
1 vote
0 answers
13 views

Testing L2 effects without any L2 variance?

I'm testing a hypothesized effect of an L2 variable on L1 outcomes (students nested in classrooms), but a few of my outcome variables have no L2 (classroom-level) variance (i.e. the within-classroom, ...
user avatar
  • 31
4 votes
0 answers
56 views

On the distribution of sample intraclass correlation for normal data

Consider i.i.d random vectors $\boldsymbol X_1,\boldsymbol X_2,\ldots,\boldsymbol X_n$ having a $p$-variate normal distribution $N_p(\boldsymbol \mu,\Sigma)$ where $\Sigma$ has compound symmetry of ...
user avatar
  • 9,120
2 votes
0 answers
43 views

Should dimensions with low reliability be omitted from EFA?

A set of items were rated on 25 dimensions by 85 participants. I've computed the ICCs on each dimension and they are all good (> .75) except one dimension which is moderate (.69) and one that is ...
user avatar
  • 1,641
2 votes
1 answer
67 views

Within-cluster correlation in lognormal shared frailty models

According to Peter Austin (ref. below), in a gamma shared frailty model (i.e. a Cox regression model with cluster specific random effects which are iid logarithms of gamma distributions), the within-...
user avatar
  • 391
0 votes
1 answer
79 views

How can I use the ICC as a measure for inter-rater agreement on a set of (more or less) correlated items?

I have a question about inter-rater agreement using the ICC. I'm trying to test, whether my study participants (the raters) rate the personality of a fictional character similarly - think of a ...
user avatar
0 votes
0 answers
65 views

Is it correct to compare ICC/VPC values between linear and logit models?

I am currently working on a research project involving the decomposition of between-group error using the intra-class correlation coefficient (ICC) which you may understand more intuitively as the ...
user avatar
0 votes
0 answers
19 views

intracluster correlation coefficient: are standard errors valid?

I am working with panel data and I want to estimate the fixed-effects model. I suspect that SE might be misleading due to (possibly) the necessity of clustering. Would the intracluster correlation ...
user avatar
  • 27
0 votes
1 answer
37 views

What's a good way to calculate agreement between examiners when the measure is binary?

I want to assess ability to identify tumor types in humans vs. a computers. There are 7 tumor types. I am interested in calculating agreement between the human examiners but I am not sure the best way ...
user avatar
  • 63
0 votes
0 answers
56 views

ICC calculation for examiner and timepoint

How do I calculate an ICC for a dataset in which there are k examiners, and each examiner takes n measurements? I know that I can use the psych package in R to calculate the ICC across the k*n ...
user avatar
  • 63
4 votes
1 answer
104 views

When are circular statistics needed (reference)?

I have a set of data of measurements for the lateral angle of the internal acoustic meatus (it's an angle between the canal of our inner ear and another bone structure in our skull). The measurements ...
user avatar
  • 105
1 vote
0 answers
47 views

ICC for cluster within cluster (survey package R)

So I'm not quite sure about the terminology but what ever it's called the actual scenario is straight-forward. I'm dealing with a two-stage cluster sample. It's also stratified but let's set that to ...
user avatar
  • 934
2 votes
1 answer
365 views

How to calculate "by hand" between group variance and the ICC for a three-level hierarchical model?

Introduction: This question is related to this and this questions that were answered before, but I would like a more detailed answer on how between-groups variance are calculated in a (null) three-...
user avatar
  • 121
2 votes
1 answer
245 views

Can pearson's correlation coefficient be converted to Cohen's kappa?

I'm interested in meta-analysing the agreement between two measures. Depending on whether the variables are continuous/categorical/binary, different studies present agreement as either: A Pearson's ...
user avatar
  • 135
0 votes
1 answer
437 views

How to interpret near-zero ICC in mixed models?

I am running an analysis of data with multiple observations per person. I assumed that therefore a mixed model analysis would be appropriate. Back then in my statistics class, we were told that ...
user avatar
  • 93
2 votes
0 answers
162 views

Interpretation of Intraclass correlation coefficient (ICC) as a reliability estimate for hierarchical designs - lme4

I am working on the following problem and would really appreciate your help: The data set at hand contains medical ratings (scale 1-5, assumed interval) by 5 independent raters, on 7 occasions, for 60 ...
user avatar
1 vote
0 answers
63 views

ICC from random effects model VS ANOVA

ICC is talked about in the context of ANOVA as well as random effects model. In case of the ANOVA approach, the calculation is defined as: ...
user avatar
  • 123
1 vote
0 answers
192 views

Why does the intraclass correlation coefficient (ICC) increase as covariates are added to a hierarchical logistic regression model?

I fit three Bayesian binary multilevel logistic regression models. Schematically they look like this: ...
user avatar
  • 389
0 votes
1 answer
44 views

Crossed or random effects? (lme4)

I am examining the effects of the type of context change (change in personnel, change in setting) on response rate of a targeted behavior (e.g., aggression) among patients in a treatment center. There ...
user avatar
  • 51
3 votes
1 answer
147 views

Measure of rater agreements for rank order?

If k raters are asked to rate the same set of objects on a continuous or Likert scale, there is the ICC3 for measuring the inter-rater agreement. Is there also an agreement measure, if all raters have ...
user avatar
  • 3,006

1
2 3 4 5 6