The Intra-class Correlation Coefficient measures the degree to which a categorical variable accounts for variability in a continuous variable. It is often used to index the degree of clustering in a dataset.

learn more… | top users | synonyms (1)

2
votes
1answer
16 views

Statistic to find out association between ordinal data

I have a 4x3 contigency table with ordinal data for a research finding in tuberculosis, I analysed cytology patterns and afb patterns; cytological pattern $A,B,C,D$ (independent variable) and afb ...
0
votes
0answers
41 views

What is a “subject” in ICC?

I have been working in R and have finally got some code working regarding intraclass correlation coefficients. In the output I get something called "subjects". I thought it was to do with the number ...
0
votes
0answers
50 views

Mean pairwise Pearson correlation

Consider a set of $M$ multidimensional observations $X=\{x_i\}_{i=1..M}$ for $x_i\in\mathbb{R}^N$, and define $c_{ij}$ to be the Pearson correlation coefficient between $x_i$ and $x_j$. Is there a ...
0
votes
1answer
30 views

Comparison between fat-matrix

I have two data sets obtained from two different kind of chemical analysis: GC-MS and IR. Both the chemical analysis were performed using the same samples. My aim is to know if there are some ...
1
vote
1answer
90 views

ICC in a linear mixed model - longnitudial data - why is this the correlation between time periods?

I'm confused by the description of the intraclass correlation (ICC) for a linear mixed model with longitudinal data from this material. A screen capture is shown below. What I'm confused about is ...
0
votes
0answers
43 views

Intraclass correlation (ICC) per item instead of across items

I am using the intraclass correlation coefficiant to determine inter-rater reliability for a set of pictures that were rated by several raters on a scale from 1-9. I am using the two-way mixed model ...
2
votes
1answer
62 views

Measuring intra-observer correlation

We're far from what you'd call expert statisticians, so we'd like to ask some help in an ongoing medical research. We have 60 subjects. A researcher took at least 3 times the measurements of 8 of ...
2
votes
1answer
159 views

How to compare repeatability (ICC) of different groups?

I have calculated the ICC values for two groups and would now like to compare the ICC values to determine if the groups differ in their repeatability. In the literature people have simply used t-tests ...
1
vote
0answers
147 views

Intra- and inter- observer error, and the intra-class correlation coefficient (ICC)

I have 3 observers, who each perform multiple observations in which they count a phenomenon. I'm trying to quantify both inter- and intra- observer error for this study. I'm new to this and am finding ...
2
votes
1answer
93 views

ICC - unclear comment from reviewer

I'm new here and not a statistician, so I apologize in advance and I sure hope someone can help! I'm working on a study where we have 10 locations $(5 \ intervention, 5 \ control)$. We have two ...
1
vote
1answer
91 views

Low intraclass correlation although between variance is very high in random effects model

I don't understand why a random effects model can give you a very low rho (intraclass correlation coefficient), if the between variance (between subjects) in your ...
1
vote
0answers
120 views

How to analyze intra-rater reliability with 31 raters who all rated 4 subjects twice

I'm trying to analyze the intra-rater reliability of an occupational therapy assessment. The assessment consists of 57 items, where some items are ratio data and some items are ordinal data. In my ...
1
vote
0answers
41 views

Averaging correlation matrices among subjects

Suppose I have N correlation coefficient matrices, containing the Pearson correlation coefficients of k different time series of ...
2
votes
0answers
22 views

Combining critic scores into single rating

I have a collection of scores from critics for various restaurants. Some restaurants are rated by multiple critics and some by only a single critic. Each critic rates restaurants using different ...
2
votes
1answer
96 views

Agreement of clustered data

I have the following situation: I have analyzed several data curves from a group of patients (16 curves per patient) with different analysis methods and want to test for the agreement of the methods. ...
3
votes
1answer
287 views

how to partition the variance explained at group level and individual level

Based on results of Null model, some scholars suggest variance attributed to group level and individual level is computed using variance component output, whereas other suggest going the standard ...
2
votes
1answer
508 views

ICC and Kappa totally disagree

Much has been written on the ICC and Kappa, but there seems to be disagreement on the best measures to consider. My purpose is to identify some measure which shows whether there was agreement between ...
2
votes
0answers
110 views

Comparing two dependent intra-class correlations (ICCs)

We are measuring two waves from a biological signal. As an example consider the ECG $P$ wave and $T$ wave. From a single trace, the amplitude of both of these waves is measured. We want to compare ...
7
votes
1answer
194 views

Intraclass correlation in the context of linear mixed-effects model

Suppose that one investigates the two sources of variability with data $y_{ij}$ acquired from $j$th subject under $i$th session ($i=1,2; j=1,2..., n$). A linear mixed-effects model can be formulated ...
1
vote
0answers
110 views

intra-rater reliability and assessment error - which ICC type to use? singular rater

I'm trying to figure out which ICC type to use to calculate MY measurement error using a particular assessment tool: background and data: PhD feasibility phase only. 10 pairs of videos of people ...
0
votes
0answers
67 views

Intraclass correlation coefficient - Can it be used for other purposes, beyond inter-rater agreement and reliability?

Can I use ICC to measure stability or change? We have data from 3 different police training practices where we have measured, for a period of 5 minutes, the number of seconds that one police officer ...
2
votes
0answers
83 views

Validity of grand mean intra-rater Kappa or ICC

For 5 raters, I have calculated intra-rater Cohen's Kappa statistics for the test-retest nominal ratings, intra-rater Kendall's tau-b statistics for the test-retest ordinal ratings, and intra-rater ...
2
votes
1answer
62 views

Looking for simple examples of how to calculate type III and type IV SS

I have data collected on five species of fish at half a dozen locations in a lake over four years. The categories are not (at all) fully crossed, and I have a lot of empty cells due to logistical ...
12
votes
2answers
443 views

Looking for Intuition Regarding Fixed/Random Effect Conundrum

I understand that we use random effects (or mixed effects) models when we believe that some model parameter(s) vary randomly across some grouping factor. I have a desire to fit a model where the ...
0
votes
2answers
449 views

What statistics should I use for evaluating the accuracy of predictions?

I have two variables representing 1) players' predicted fantasy football points and 2) players' actual fantasy football points scored. What statistics are best for assessing the accuracy of the ...
2
votes
0answers
381 views

Computing Cronbach's Alpha and Intraclass Correlation coefficient from an lmer model

I am trying to calculate the Intraclass Correlation for a rater study using R and the library lme4 and the function lmer. The data has the following design: The same 6 raters (at least 4) are rating ...
1
vote
2answers
568 views

Several intraclass correlations (ICC) when modeling heterogeneous variances?

In a nested design (e.g., students in schools, ants in colonies, individuals in a species, leaves in a tree), ICC is defined as the proportion of variance among observations from the same subject ...
2
votes
0answers
82 views

Assessing the ability of two quantitative methods measuring the same thing

Suppose I have two scientific approaches to measuring the same physical quantity taking values between $[0,1]$. One (say method A) is generally accepted to be more accurate under certain conditions, ...
3
votes
0answers
1k views

For intraclass correlation coefficients (ICC), interpret both “single” and “average” measured ICC values for reproducibility test?

Say there's a new machine for use in testing a biochemical marker's concentration in the blood, and we would like to know the reproducibility of this test. We draw blood samples from 10 volunteers, ...
0
votes
1answer
448 views

Intraclass correlation coefficient, non parametric data

Does the ICC require that my data are normally distributed? If more information about my data is needed, just ask.
1
vote
0answers
108 views

Intrarater reliability

I would like to estimate the intrarater reliability in a MRI study. Different regions of interest were manually traced with a mouse-driven cursor in order to estimate their volumes. All data were ...
3
votes
2answers
393 views

Inconsistency of findings between Kappa and ICC for IRR study

I used the following R script to calculate the Kappa and ICC for a measured done by two raters on 23 subjects: ...
6
votes
0answers
227 views

What GEE-exchangeable method can do that robust variance can't?

I asked a related question before here on the difference between GEE method with exchangeable varcov structure v. Robust standard errors known as Huber White method in group randomized trials. As ...
5
votes
1answer
460 views

How to statistically test whether there is any nesting effect in the data?

I have a sample including all students within 52 schools which have been randomly assigned to be treatment or control (we have group randomized trial in which all students in each school is either ...
7
votes
1answer
836 views

How to perform inter-rater reliability with multiple raters, different raters per participant, and possible changes over time?

Participants were rated twice, with the 2 ratings separated by 3 years. For most participants the ratings were done by different raters, but for some (< 10%) the same rater performed both ratings. ...
0
votes
0answers
311 views

Nonparametric ICC

Anyone can suggest me a nonparametric version of the intraclass correlation coefficient (ICC)? My model have the following details: two-way random effects model, consistency, average ratings as unit. ...
1
vote
0answers
99 views

Intraclass correlation to validate a measure

I would like to understand whether the output of an algorithm I developed agrees with the ratings of external observers. The number of observers is 10 and each of them rated on 2 3-point-Likert items ...
1
vote
0answers
141 views

Within group correlation of pearson's residuals

I have social network data in which an "ego" names a friend "alter". I am running a regression in R in which attributes of alter are predictors of outcomes for ego. So each observation is dyadic with ...
1
vote
1answer
2k views

Justification of aggregation in multi-level model - ICC in SPSS

I am running a multi-level model on data which is set out with one row per individual survey return, each with Individual ID and Organisational ID as columns. There are variables which can be ...
1
vote
0answers
108 views

How to compute effect size for different measures of association?

My colleagues and I are looking for guidance on how we might compute a common effect size (ES) from the following measures of association: Pearson's r Spearman's rho Kappa ICC / intraclass ...
1
vote
0answers
585 views

How would one calculate intrarater reliability in Stata?

My question follows from a previous excellent response in another post: Intra- and inter-rater reliability on the same data Wow, what a truly great response! I am running through a similar problem ...
0
votes
1answer
107 views

Correlated errors between variables [between / within subjects, within variables, everything]!

I am trying to figure out good model to analyze social dynamics using a model. I have commenting data from a website. Such as 500 articles, 10000 users, and their 75000 comments. Every user does not ...
2
votes
1answer
436 views

ICC reduced to 0 in a multi-level model. Should this be suspicious?

I calculated a sequence of multi-level models with stata (xtmixed). I have a sample of 800 kids in 46 classes and I want to explain their individual level of prejudice by the proportion of fully ...
1
vote
1answer
290 views

Assumptions for intraclass correlation

I am having difficulty finding information on the assumptions for the intraclass correlation. Can someone please tell me what they are?
14
votes
1answer
1k views

Intraclass correlation (ICC) for an interaction?

Suppose I have some measurement for each subject at each site. Two variables, subject and site, are of interest in terms of computing intraclass correlation (ICC) values. Typically I would use ...
20
votes
1answer
905 views

Computing repeatability of effects from an lmer model

I just came across this paper, which describes how to compute the repeatability (a.k.a. reliability, a.k.a. intraclass correlation) of a measurement via mixed effects modelling. The R code would be: ...
7
votes
0answers
615 views

Intraclass Correlation Coefficients (ICC) with Multiple Variables

Suppose I have measured some variable in siblings, which are nested within families. The data structure looks like this: family sibling value ------ ------- ----- 1 1 y_11 1 2 ...
4
votes
1answer
3k views

Intra- and inter-rater reliability on the same data

I've read some of the answers here (specifically the 'Inter-rater reliability for ordinal or interval data' one). I'm still somewhat perplexed though! I have data for 4 raters, who each rated every ...
7
votes
2answers
983 views

Comparing correlation coefficients

I have two sets of data where I have at ~250.000 values for 78 and 35 samples. Some of the samples are members of a family and this may have an effect of the data. I have calculated pairwise ...
3
votes
2answers
185 views

Specification of a random slopes model with two grouping factors

Suppose people take a 10-item exam. For each item, $k = 1,2,…,10$, exactly one rater assigns a score to exactly one person, with the constraint that no person sees the same rater twice. There are $i = ...