Questions tagged [intraclass-correlation]

Intraclass correlation coefficient (ICC) measures how much quantitative values are similar or associated within groups. There exist several versions and alternative formulations of ICC. The correlation is used to assess a dataset clusteredness, agreement among raters and in other settings.

Filter by
Sorted by
Tagged with
0
votes
0answers
17 views

Differences between glmmadaptive Vs lme4 and glmmTMB in ICC measurement

This is my first question, so please be kind... I am currently modelling a GLMM with a binary outcome with many (500+) clusters but cluster size of 2 (by design - there can be no more than 2 per ...
0
votes
0answers
7 views

How to calculate ICC for a zero-inflated negative binomial model

I did a zero-inflated negative binomial regression on some data. However, the data is nested (students within schools), and I would like to calculate ICC to make sure that we did not need to take this ...
0
votes
0answers
13 views

Getting significant between effects in mediation despite low ICC

I am interested in examining between effects using lmer in a mediation model using mediation and within effects using ...
4
votes
0answers
40 views

Meta-analyzing ICCs to avoid using multiple random effects?

(Note: Parts where I've encountered problems are in bold to make them easily findable.) I have data from a longitudinal study of twins. Several biomarkers are measured, usually 4 measurements each (...
0
votes
0answers
7 views

How should I compare variability across repeated measures in a single trial vs variability across measurements across repeated trials?

I have collected some data from different subjects in 2 different ways. In the first way, each subject performed one long trial, I segmented the data captured and and calculated a variable value for ...
0
votes
0answers
13 views

Should I use ANOVA or ICC to demonstrate no effect between measurements before and after a treatment?

I have taken a performance measurement on a group of subjects, then put those subjects through a treatment, and remeasured their performance. I would like to comment on the similarity between the ...
4
votes
0answers
159 views

Intraclass correlation coefficient (ICC) for two raters using a mixed effects model for a design with repeated measurements

I want to calculate the intraclass correlation coefficient (ICC) for an inter-rater assessment with two raters. The design is as follows: Each of the $n$ subjects were assessed by the same 2 raters on ...
0
votes
0answers
21 views

Which ICC (ICC(2,1) or ICC(2,k)) should I report?

I have some hundred subjects indicating on a 7point scale for 10 situations how likely they are to be influenced in diverse situations. I will build an index of the responses for these behaviors and ...
0
votes
1answer
23 views

How to calculate intra-observer error and average difference?

I am trying to assess intra-observer error in a setting where measurements where done repeatedly by a single observer. So far I'm having some conceptual and technical issues. In the dataset we have ...
0
votes
1answer
27 views

Is the ICC appropriate here?

I have N tissue samples, labelled as either A or B. 2 measurements are taken from each sample. The data is not normally distributed. I'm interested in 2 things: firstly, whether there is ...
0
votes
0answers
13 views

How to compute intraclass correlation (ICC) for THREE-level logit hierarchical model?

I know that for a two level model the formula is: ICC = 𝜎^2/(𝜎^2+(π^2/3) But how can I calculate it for a three level logit MLM?
1
vote
0answers
12 views

Measuring equivalence/reliability (ICC and consorts)

For a reliability study I’m administering patient reported outcomes (questionnaires) with 2 modes of administration (on paper and via an electronic device). I’d like to measure equivalence/reliability ...
0
votes
0answers
17 views

Calculating ICC using z-scored standardized or unstandardized DV in Multilevel Linear Model?

I am doing multilevel linear modelling and I am calculating my ICC for my random intercept model. However, when I use the z-scored standardized DV (reaction time), the intercept and residual variance ...
0
votes
0answers
250 views

Interpretation of low intraclass correlation coefficient (ICC) values and assumptions of independence

I have an unbalanced repeated measures data set and my main goal is to see what stats are correlated with win percentage. There is an unbalanced number of observations per player. With my data set I ...
0
votes
0answers
7 views

Measure stability of radiomic features using ICC

I have a question regarding ICC. I want to measure how different MRI T1 pulse sequences affect radiomic features extraction. I have at the moment 7 patients, all of them have T1 SE and T1 GRE scans. ...
0
votes
0answers
30 views

Checking ICC in MLM when DV is level 2

I'm running a multilevel model in SPSS where my data is grouped by teams. The DV is a level 2 variable since the team as a whole got a score (1 - 10) for completing a task. The IV is level 1 such that ...
0
votes
0answers
21 views

Is it okay to use ICC with auto-correlated time series data?

Is it okay to use ICC with autocorrelated data and if not, how do you deal with this? I'm trying to measure reliability of devices measuring a continuous physiological signal.
0
votes
1answer
128 views

What are p and p0 in ICC.Sample.Size

The purpose of this R package should be self-explanatory. However, its parameters are not. The help page never explains what is meant by p and ...
1
vote
1answer
55 views

interrater-reliability with likert-scale and different raters

I am collecting data from a survey that includes a likert scale of 5 points. Each rater answers 18 questions (probably different questions). For each question I will have answers from 10 different ...
0
votes
0answers
42 views

How to calculate Intraclass Correlation Coefficients (ICC) for hierarchical or clustered data

I have 4 measurements for each of my 1000 subjects, and each subject has a bunch of parameters (p1, p2, ... p100), for each of which I would like to calculate an ICC (version: (3,1) according to ...
0
votes
0answers
281 views

How should clustering be accounted for in logistic regression, when there are very few clusters?

I have survey data from 1000 patients. This is a convenience venue-based sample. In a specific city, at 9 hospitals that happen to have a psychosocial program, patients can opt into the program if ...
1
vote
0answers
10 views

Should reviews/ratings data be considered as repeated measures?

Suppose I have some ratings/review data for a type of product (say coffee). There are many different kinds of coffee and there are many different subjects reviewing coffee and also some covariates ...
0
votes
0answers
128 views

Intraclass Correlation Coefficient in multilevel models (HLM)

I am currently writing my Master's Thesis and struggling to find a convenient way of calculating the ICC for my multilevel model (random intercept, fixed slopes). Information to the data: I have ...
2
votes
0answers
126 views

Adjust intraclass correlation for measurement error

It's been known since Spearman (1904) that one can adjust an observed Pearson correlation for measurement error using this formula (R code with tidyverse attached): ...
0
votes
1answer
102 views

icc in hierarchical models as variance partition coefficient

When I learned multilevel modeling, one of the first things we were required to do is to run an empty model with the dependent variable and the clustering IDs. In my understanding, the ICC of such a ...
0
votes
0answers
97 views

Can ICC be used to measure and benchmark ordinal classification accuracy?

I am conducting ordinal regression with an ordinal categorical target outcome roughly equally distributed as 1, 2 or 3. (This ordinal categorical target is actually the conversion of a numeric target ...
1
vote
1answer
38 views

Comparing dependent correlations

I have a simple single-group pre-post design and have Pearson correlations for performance and self-report for both times of measurement for each participant that I would like to compare. Would using ...
0
votes
1answer
124 views

Logistic linear mixed effects model with categorical predictors

I've read a lot of threads on stack exchange but haven't exactly found what I'm looking for. Everyone seems to have a slightly different problem/issue. First, lets have a look at my data: 120 Users ...
0
votes
1answer
62 views

Intraclass Correlation Coefficient with Bayesian ordered-logit GLMM (STAN)

I am fitting a Generalized Linear Mixed Model for an ordered outcome, in form of an ordered logit, with random intercept and slope. For this task, I am going Bayesian by handling STAN through the ...
1
vote
1answer
415 views

Interpretation of ICC when multiple random intercepts and fixed effects are collinear in mixed models

The intraclass correlation coefficient (ICC) is used in mixed models to give a sense of how much variance is explained by a random effect. It is calculated by dividing the variance of the random ...
0
votes
1answer
642 views

Calculating ICC 2.1C in R

Intraclass Correlations using Mcgraw and Wong conventions defines 5 ICC´s for single scores. I am interested in calculating both Two-way random, single measures, absolute agreement (Sometimes ...
0
votes
0answers
47 views

Disagreeing measures of agreement for multilevel modeling

I created a questionnaire of 3 parts. Each part has a few questions. I now want to see if I can gather data into level 2 modeling (i.e. take each part and gather all answers to have only one number). ...
1
vote
1answer
38 views

Intraclass Correlations for Intraobserver-Reliability in Hierarchical Data

We have a dataset where multiple measurements of the same variable are taken from individuals and we want to measure the intraobserver reliability. However, the individuals each come from different ...
0
votes
0answers
149 views

Test-Retest reliability for mixed models

We have an experimental setup as follows: A set of subjects each repeat a single task multiple times in a single session, yielding a set of measurements. The same subjects then repeat the same task ...
1
vote
0answers
23 views

best statistical approach to study the time evolution of clustering in a data set

I am using a stochastic method for the clustering of a data set. The number of clusters that this approach returns, can differ in each iteration. On the other hand, I would like to study the evolution ...
0
votes
1answer
22 views

How to test correlation in this model

I have a model with $n$ participants. From 5 different angles $\theta_1, \theta_2, \dots, \theta_5$, each participant either responds to a stimulant with either a $0$ or a $1$, so $y_{i,\theta_j} \in \...
1
vote
1answer
188 views

Calculating variance components and ICC of a random intercept model by hand

When I really want to understand a measure or parameter, I tend to do the calculation by hand with simplified data. Today I have attempted to do the same with the ICC, but somehow keep failing. I was ...
1
vote
0answers
35 views

Intra-class correlation between two raters

I'm reading a paper that compares two raters (one human, one algorithm) that rate images with a single number (total volume of an object in the image), and there are about 100 images. Each image is an ...
1
vote
0answers
31 views

Measuring emotion differentiation on a within-person level.

To measure emotion differentiation, which refers to the precision of an individual to accurately describe and distinguish between different emotional states, many studies have used a between-person ...
-1
votes
1answer
1k views

When do I use two-way mixed, two way random and one way random for the Intraclass correlation coefficient?

I have a question about the intraclass correlation coëfficiënt. I have 62 patients for whom I calculated the breast tumor volume. The surgeon who resected the tumors did the same thing, but without ...
-1
votes
1answer
206 views

Adjusted ICC with mixed effects model in R [closed]

I am trying to run an intraclass correlation for a small dataset, adjusting for the interval of time between sessions, which varied across participants. In the data below, value is the dependent ...
2
votes
0answers
151 views

Possible to convert intraclass correlation coefficients ($ICC$) for use in meta-analysis of $r$?

I am trying to meta-analyze the validity evidence for a variety of new-fangled measurement devices. My effect size of interest is the correlation (Pearson's $r$) between measurements from these new ...
1
vote
1answer
34 views

Which is the state of the art regarding methods for Association rule learning?

I have a set of data about which I would like as a first step, discover some interesting or frequent relationships. For this I know that there are several algorithms, like association or sequence ...
2
votes
1answer
120 views

Which ICC model to use for computer vs. human agreement

I have an agreement study with the following basic construction: ~ 400 items are rated by (1) a computer algorithm, and (2) a human rater. The ratings are on a continuous scale. I would like to ...
0
votes
1answer
384 views

Is high ICC essential when using HLM?

I have a question related to 'whether HLM is necessary or not.' My data has regions and firms with the time range of 2005 and 2010. I set each firm (with multiple observations within it) as the ...
0
votes
1answer
54 views

How to correlate time series to a variable

In an experiment, heart rate of participants is collected for 60 seconds (1 reading per seconds) and pupil diameter whilst performing one of three tasks. The success of the task is measured with a ...
1
vote
0answers
49 views

reliability of a device for my thesis and negative icc values in spss

first time here , hoping for a great answer! so my thesis is including using a sensor to measure how far is a a person leg is from the sensor. so i measured it with a ruler and compared with sensor ...
0
votes
1answer
250 views

ICC low, but still team variables in individual model?

I have data on two levels: team and individual. However, since the ICC on team level is low, I’ll not use a multilevel model but just a regular regression on individual level. After all, the impact of ...
1
vote
0answers
32 views

What test can I use for statistical significance?

This is a graph of the distributions of population proportion (y-axis) by population density (x-axis) for four different modal ages of housing stock. The black line, which we are interested in, ...
2
votes
0answers
33 views

Low ICC for Healthy but Large Difference from Diseased

I have a test I am interested in further investigating for diagnostic purposes. The test involves 6 trials, giving a mean result and variance for each person. What we have found through Mann-Whitney ...