Questions tagged [intraclass-correlation]

Intraclass correlation coefficient (ICC) measures how much quantitative values are similar or associated within groups. There exist several versions and alternative formulations of ICC. The correlation is used to assess a dataset clusteredness, agreement among raters and in other settings.

Filter by
Sorted by
Tagged with
1
vote
0answers
13 views

What's the difference between coefficient of variation and intraclass coefficient and when should I use one vs the other?

If I am trying to compare variability in a measurement taken multiple times in a single session with the variability in that measurement taken over multiple sessions, should I use icc or cov? I am ...
0
votes
0answers
9 views

Can I use ICC (Intraclass Correlation) as a decision criteria to use a model with random intercepts or a model with random intercepts and slopes?

I am thinking on using a ICC as a decision criteria for choosing a mixed model with different random structures in r ...
1
vote
0answers
29 views

Calculating interclass correlation but with different observer experience

I have tried finding an answer to the following question both here and using PubMed and google but without luck. I have been asked to calculate the intraclass correlation coefficient from the ...
2
votes
0answers
21 views

ICC and power for mixed models

I'm simulating data for a mixed model in order to estimate the needed power for a study. I have 3 groups, a control group and two treatment groups. The effect sizes for the two treatment groups are 0....
1
vote
0answers
24 views

Modelling how test agreement varies as a function of a covariate

I have a group of human subjects who have been given two versions of a test measuring the same outcome. I am interested in the agreement between these two tests, and whether their agreement varies as ...
0
votes
0answers
10 views

Intra-Rater Agreement of Dichotomous Variables in Qualitative Research

Given a set of 21 interviews and a codebook (60 codes), each coder decides on whether a code should be set (1) or not (0) in order to retrieve a (binary) feature vector describing the content of the ...
1
vote
0answers
39 views

ICC Value of 0. Boundary singular fit? Interpretation?

In this case I have 7 samples in which the concentration of a particular analyte was measured and the measurement was repeated 3 times for each sample. Here's the data ...
1
vote
1answer
42 views

How is clustered data and correlated data related?

I am confused about the connection between clustered data and correlated data. Let's say we have a two-level data where level 1 refers to students and level 2 refers to schools. I assume we can also ...
1
vote
1answer
26 views

Why is the equation for Intra-class correlation aligned with design effect of cluster sampling?

The design effect definition (link) is: $$ DEFF = 1+ (m-1)*ICC $$ It is also the relationship between the variance of estimating the mean with the cluster sampling divided by the variance if the ...
2
votes
0answers
17 views

Multilevel Analysis for count variable

I have a general question about multilevel analysis and have therefore created a minimal example to represent the data I want to analyse: ...
1
vote
0answers
24 views

To pool or combine multiple correlations: Pearson and ICC

For part of a study we are running, we are investigating the correlation between a post-practice score, and the subsequent day's pre-practice score. We are looking at the potential association between ...
0
votes
0answers
18 views

How low should an ICC be to justify disaggregating (i.e. ignore clustering) a multilevel/mixed model?

It is conventional wisdom that ICCs can be used to estimate how much clustering/non-independence is occurring in data consisting of two or more levels. ICCs can inform whether a multilevel/mixed model ...
1
vote
0answers
9 views

Asymptotic distribution of intraclass correlation

Consider the random effects ANOVA model below (notation based on Snijders and Bosker, 1999), where $j$ represents a group and $i$ an individual: $$ Y_{ij} = \mu + U_j + R_{ij}, \qquad var(Y_{ij}) = \...
0
votes
1answer
15 views

Fluctuation and standard errors in election polling

British politics is very interesting lately; particularly so since Corbyn was elected Labour leader in 2015, and later with Brexit happening and so on. Since then, multiple events have had dramatic ...
0
votes
1answer
36 views

Inter-rater reliability or intraclass correlation for one subject

I'm looking for a way, in R, to get measures of agreement for each item. I have 32 ordinal items (we want separate statistics for each), and 66 raters.I have two subjects, but we want to look at the ...
0
votes
0answers
39 views

How to use ICC(intraclass correlation coefficient) to test the correlation between categorical and numerical variables?

I have a dataset with both numerical and categorical variables.I'm trying to test the correlation or say, strength of association between these two kinds of variables. I already tried the eta and eta ...
2
votes
0answers
38 views

Interpreting ICC in 3-level logistic regression

I've run a 3-level logistic regression in STATA- level one is maternal health clinics (N=200), level 2 is the primary health care centres(N=41) which run the clinics and level 3 is the district(N=8). ...
2
votes
2answers
129 views

Intra-observer variance versus inter-observer variance for non-i.i.d. measurements

I have three judges (A,B,C) that estimate a value for N objects. Each judge estimates the same object three times, so that my data frame looks like ...
0
votes
1answer
23 views

How to do Intraclass correlations & Within-group reliability in SPSS?

Friends, My sincere apologies if this question has been posted earlier. I am looking for help in terms of how to do Intraclass Correlations and find out how to do within group reality in SPSS. I am ...
0
votes
0answers
16 views

What statistical test should I use to check that test-retest data are not significantly different in a repeated measures design?

I have a repeated measures study with two independent factors. The first factor is an environment factor that with three environments: a, b, and c. The second factor is the treatment: no treatment, ...
0
votes
0answers
60 views

Intra-class Correlation Coefficient (ICC) for a single subject

I need to calculate a score that reflects the level of disagreement/ambiguity between raters for a music clip. The experimental set-up is that 10 respondents listen to a clip with a one-dimensional ...
0
votes
0answers
18 views

Bayesian Comparison of Dependent Correlation Coefficients

I would like to compare the correlation coefficients from two different models of the same data points. One of them is the raw bivariate data (Value vs Time), and the other is given a biologically-...
0
votes
0answers
44 views

Variance Partitioning Coefficient to help random effect interpretation in lme4 model with both random intercepts and slopes?

I am working on a project having to do with disadvantaged families accessing support services from a charity. The structure of the data places us within a mixed model context: for the random side, we ...
0
votes
0answers
28 views

Test-Retest reliability/ICC Help?

I’m sure this is a simple question but I’m new to statistics and would really appreciate the help! The study I am working collected data over two testing days. On each testing day subjects ran a ...
0
votes
0answers
11 views

Inter and Intra-rater reliability

I am working on a project, in which we asked 5 raters to evaluate a bone dyplasia. All raters evaluated the dysplasia between o and 3. 0: normal 1: mild dysplasia 2: moderate dysplasia 3: severe ...
0
votes
0answers
132 views

Differences between glmmadaptive Vs lme4 and glmmTMB in ICC measurement

This is my first question, so please be kind... I am currently modelling a GLMM with a binary outcome with many (500+) clusters but cluster size of 2 (by design - there can be no more than 2 per ...
0
votes
0answers
27 views

How to calculate ICC for a zero-inflated negative binomial model

I did a zero-inflated negative binomial regression on some data. However, the data is nested (students within schools), and I would like to calculate ICC to make sure that we did not need to take this ...
0
votes
0answers
15 views

Getting significant between effects in mediation despite low ICC

I am interested in examining between effects using lmer in a mediation model using mediation and within effects using ...
4
votes
0answers
47 views

Meta-analyzing ICCs to avoid using multiple random effects?

(Note: Parts where I've encountered problems are in bold to make them easily findable.) I have data from a longitudinal study of twins. Several biomarkers are measured, usually 4 measurements each (...
0
votes
0answers
13 views

How should I compare variability across repeated measures in a single trial vs variability across measurements across repeated trials?

I have collected some data from different subjects in 2 different ways. In the first way, each subject performed one long trial, I segmented the data captured and and calculated a variable value for ...
0
votes
0answers
39 views

Should I use ANOVA or ICC to demonstrate no effect between measurements before and after a treatment?

I have taken a performance measurement on a group of subjects, then put those subjects through a treatment, and remeasured their performance. I would like to comment on the similarity between the ...
5
votes
1answer
350 views

Intraclass correlation coefficient (ICC) for two raters using a mixed effects model for a design with repeated measurements

I want to calculate the intraclass correlation coefficient (ICC) for an inter-rater assessment with two raters. The design is as follows: Each of the $n$ subjects were assessed by the same 2 raters on ...
0
votes
0answers
68 views

Which ICC (ICC(2,1) or ICC(2,k)) should I report?

I have some hundred subjects indicating on a 7point scale for 10 situations how likely they are to be influenced in diverse situations. I will build an index of the responses for these behaviors and ...
0
votes
1answer
149 views

How to calculate intra-observer error and average difference?

I am trying to assess intra-observer error in a setting where measurements where done repeatedly by a single observer. So far I'm having some conceptual and technical issues. In the dataset we have ...
0
votes
1answer
34 views

Is the ICC appropriate here?

I have N tissue samples, labelled as either A or B. 2 measurements are taken from each sample. The data is not normally distributed. I'm interested in 2 things: firstly, whether there is ...
0
votes
0answers
43 views

How to compute intraclass correlation (ICC) for THREE-level logit hierarchical model?

I know that for a two level model the formula is: ICC = 𝜎^2/(𝜎^2+(π^2/3) But how can I calculate it for a three level logit MLM?
1
vote
0answers
19 views

Measuring equivalence/reliability (ICC and consorts)

For a reliability study I’m administering patient reported outcomes (questionnaires) with 2 modes of administration (on paper and via an electronic device). I’d like to measure equivalence/reliability ...
0
votes
0answers
22 views

Calculating ICC using z-scored standardized or unstandardized DV in Multilevel Linear Model?

I am doing multilevel linear modelling and I am calculating my ICC for my random intercept model. However, when I use the z-scored standardized DV (reaction time), the intercept and residual variance ...
0
votes
0answers
593 views

Interpretation of low intraclass correlation coefficient (ICC) values and assumptions of independence

I have an unbalanced repeated measures data set and my main goal is to see what stats are correlated with win percentage. There is an unbalanced number of observations per player. With my data set I ...
0
votes
0answers
12 views

Measure stability of radiomic features using ICC

I have a question regarding ICC. I want to measure how different MRI T1 pulse sequences affect radiomic features extraction. I have at the moment 7 patients, all of them have T1 SE and T1 GRE scans. ...
0
votes
0answers
44 views

Checking ICC in MLM when DV is level 2

I'm running a multilevel model in SPSS where my data is grouped by teams. The DV is a level 2 variable since the team as a whole got a score (1 - 10) for completing a task. The IV is level 1 such that ...
0
votes
0answers
42 views

Is it okay to use ICC with auto-correlated time series data?

Is it okay to use ICC with autocorrelated data and if not, how do you deal with this? I'm trying to measure reliability of devices measuring a continuous physiological signal.
0
votes
1answer
255 views

What are p and p0 in ICC.Sample.Size

The purpose of this R package should be self-explanatory. However, its parameters are not. The help page never explains what is meant by p and ...
1
vote
1answer
114 views

interrater-reliability with likert-scale and different raters

I am collecting data from a survey that includes a likert scale of 5 points. Each rater answers 18 questions (probably different questions). For each question I will have answers from 10 different ...
0
votes
0answers
846 views

How should clustering be accounted for in logistic regression, when there are very few clusters?

I have survey data from 1000 patients. This is a convenience venue-based sample. In a specific city, at 9 hospitals that happen to have a psychosocial program, patients can opt into the program if ...
2
votes
0answers
11 views

Should reviews/ratings data be considered as repeated measures?

Suppose I have some ratings/review data for a type of product (say coffee). There are many different kinds of coffee and there are many different subjects reviewing coffee and also some covariates ...
2
votes
0answers
153 views

Adjust intraclass correlation for measurement error

It's been known since Spearman (1904) that one can adjust an observed Pearson correlation for measurement error using this formula (R code with tidyverse attached): ...
0
votes
1answer
210 views

icc in hierarchical models as variance partition coefficient

When I learned multilevel modeling, one of the first things we were required to do is to run an empty model with the dependent variable and the clustering IDs. In my understanding, the ICC of such a ...
1
vote
1answer
48 views

Comparing dependent correlations

I have a simple single-group pre-post design and have Pearson correlations for performance and self-report for both times of measurement for each participant that I would like to compare. Would using ...
0
votes
1answer
379 views

Logistic linear mixed effects model with categorical predictors

I've read a lot of threads on stack exchange but haven't exactly found what I'm looking for. Everyone seems to have a slightly different problem/issue. First, lets have a look at my data: 120 Users ...

1 2 3 4 5