Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [intraclass-correlation]

Intraclass correlation coefficient (ICC) measures how much quantitative values are similar or associated within groups. There exist several versions and alternative formulations of ICC. The correlation is used to assess a dataset clusteredness, agreement among raters and in other settings.

0
votes
0answers
23 views

How should clustering be accounted for in logistic regression, when there are very few clusters?

I have survey data from 1000 patients. This is a convenience venue-based sample. In a specific city, at 9 hospitals that happen to have a psychosocial program, patients can opt into the program if ...
1
vote
0answers
9 views

Should reviews/ratings data be considered as repeated measures?

Suppose I have some ratings/review data for a type of product (say coffee). There are many different kinds of coffee and there are many different subjects reviewing coffee and also some covariates ...
0
votes
0answers
16 views

Intraclass Correlation Coefficient in multilevel models (HLM)

I am currently writing my Master's Thesis and struggling to find a convenient way of calculating the ICC for my multilevel model (random intercept, fixed slopes). Information to the data: I have ...
2
votes
0answers
89 views

Adjust intraclass correlation for measurement error

It's been known since Spearman (1904) that one can adjust an observed Pearson correlation for measurement error using this formula (R code with tidyverse attached): ...
0
votes
1answer
29 views

icc in hierarchical models as variance partition coefficient

When I learned multilevel modeling, one of the first things we were required to do is to run an empty model with the dependent variable and the clustering IDs. In my understanding, the ICC of such a ...
-1
votes
0answers
8 views

Which test to assess the degree of concordance between a childs response and their parent response to a questionnaire

I'm looking to assess whether the difference between a child's response, and their parent's response, to a nearly identical questionnaire is significant or not. Some papers have used Pearson’s ...
0
votes
0answers
17 views

Can ICC be used to measure and benchmark ordinal classification accuracy?

I am conducting ordinal regression with an ordinal categorical target outcome roughly equally distributed as 1, 2 or 3. (This ordinal categorical target is actually the conversion of a numeric target ...
1
vote
1answer
30 views

Comparing dependent correlations

I have a simple single-group pre-post design and have Pearson correlations for performance and self-report for both times of measurement for each participant that I would like to compare. Would using ...
0
votes
1answer
38 views

Logistic linear mixed effects model with categorical predictors

I've read a lot of threads on stack exchange but haven't exactly found what I'm looking for. Everyone seems to have a slightly different problem/issue. First, lets have a look at my data: 120 Users ...
0
votes
0answers
31 views

Negative value of ICC coefficient in SPSS. Relationship among two experiments

I performed a fatiguing experiment on human muscles with 13 healthy subjects. The subjects performed the experiment till task failure and i took RMS and mean frequency of sEMG signals. then (after ...
0
votes
1answer
30 views

Intraclass Correlation Coefficient with Bayesian ordered-logit GLMM (STAN)

I am fitting a Generalized Linear Mixed Model for an ordered outcome, in form of an ordered logit, with random intercept and slope. For this task, I am going Bayesian by handling STAN through the ...
0
votes
0answers
12 views

cross-level interaction significant with three levels but insignificant with two levels

Help appreciated! I have run a multilevel model with two levels (students-classes) in which a cross level interaction appears insignificant (an interaction between a student-level predictor and a ...
1
vote
1answer
99 views

Interpretation of ICC when multiple random intercepts and fixed effects are collinear in mixed models

The intraclass correlation coefficient (ICC) is used in mixed models to give a sense of how much variance is explained by a random effect. It is calculated by dividing the variance of the random ...
0
votes
1answer
96 views

Calculating ICC 2.1C in R

Intraclass Correlations using Mcgraw and Wong conventions defines 5 ICC´s for single scores. I am interested in calculating both Two-way random, single measures, absolute agreement (Sometimes ...
0
votes
0answers
20 views

Reliability in health and disease

I have a dataset consisting of 2 groups, healthy subjects and patients. Both groups did the same test twice. The groups are not even (n=20 and n= 39 respectively). I want to get the reliability of my ...
0
votes
0answers
19 views

Disagreeing measures of agreement for multilevel modeling

I created a questionnaire of 3 parts. Each part has a few questions. I now want to see if I can gather data into level 2 modeling (i.e. take each part and gather all answers to have only one number). ...
1
vote
0answers
16 views

Intraclass Correlations for Intraobserver-Reliability in Hierarchical Data

We have a dataset where multiple measurements of the same variable are taken from individuals and we want to measure the intraobserver reliability. However, the individuals each come from different ...
0
votes
0answers
81 views

Test-Retest reliability for mixed models

We have an experimental setup as follows: A set of subjects each repeat a single task multiple times in a single session, yielding a set of measurements. The same subjects then repeat the same task ...
1
vote
0answers
20 views

best statistical approach to study the time evolution of clustering in a data set

I am using a stochastic method for the clustering of a data set. The number of clusters that this approach returns, can differ in each iteration. On the other hand, I would like to study the evolution ...
0
votes
0answers
14 views

Measures of rater agreement for timing dataset

I have a dataset that contains multiple raters. Each rater had to watch 6 videos and indicate the time at which an event occurred (e.g., 2 secs, etc.) during each trial (60 trials total). For example: ...
0
votes
0answers
39 views

Multiple Imputation of Multilevel data

I am using Mice package in R for multiple imputation of a multilevel data where repeated measures are nested within individuals. But there is a bug in mice for which we need to convert the group ...
0
votes
1answer
22 views

How to test correlation in this model

I have a model with $n$ participants. From 5 different angles $\theta_1, \theta_2, \dots, \theta_5$, each participant either responds to a stimulant with either a $0$ or a $1$, so $y_{i,\theta_j} \in \...
0
votes
0answers
16 views

finding intraclass correlation in a set of unequally sized groups

I'm trying to analyze a set of N samples taken from M subjects, with N>M (each subject was sampled at least once). I'm interested in finding the intraclass correlation coefficient (ICC). After looking ...
0
votes
0answers
29 views

Interpreting variance given stepwise changes in generalized linear mixed effects models (glmer in lme4 for R)

I am conducting an analysis for which I would like to better quantitatively understand how the incremental introduction of additional variables to mixed effects models can be used to infer which ...
0
votes
1answer
50 views

Calculating variance components and ICC of a random intercept model by hand

When I really want to understand a measure or parameter, I tend to do the calculation by hand with simplified data. Today I have attempted to do the same with the ICC, but somehow keep failing. I was ...
0
votes
0answers
15 views

Is Intra-cluster correlation recommendable to test an imputation method?

I am thinking at a situation where we are testing the performance of an imputation method, by using real values. Is it ok to use Intra-cluster correlation, by considering each value-estimate pair as a ...
1
vote
0answers
26 views

Intra-class correlation between two raters

I'm reading a paper that compares two raters (one human, one algorithm) that rate images with a single number (total volume of an object in the image), and there are about 100 images. Each image is an ...
0
votes
0answers
13 views

How to calculate momentary intra class correlations (ICC)

I came across this paper (http://dx.doi.org/10.1037/abn0000064 )that has intensive longitudinal data (128 measurement points per participant). They are interested in momentary negative affect ...
1
vote
0answers
23 views

Measuring emotion differentiation on a within-person level.

To measure emotion differentiation, which refers to the precision of an individual to accurately describe and distinguish between different emotional states, many studies have used a between-person ...
-1
votes
1answer
591 views

When do I use two-way mixed, two way random and one way random for the Intraclass correlation coefficient?

I have a question about the intraclass correlation coëfficiënt. I have 62 patients for whom I calculated the breast tumor volume. The surgeon who resected the tumors did the same thing, but without ...
-1
votes
1answer
98 views

Adjusted ICC with mixed effects model in R [closed]

I am trying to run an intraclass correlation for a small dataset, adjusting for the interval of time between sessions, which varied across participants. In the data below, value is the dependent ...
0
votes
0answers
9 views

Estimating (net?) effect of categorical predictors on a continuous measure

I don't have a lot of advanced statistics knowledge so I don't know if my summary is using accurate terms. But here's my problem. I have survey result data. 60 questions by 60 people gives 3600 rows, ...
2
votes
0answers
108 views

Possible to convert intraclass correlation coefficients ($ICC$) for use in meta-analysis of $r$?

I am trying to meta-analyze the validity evidence for a variety of new-fangled measurement devices. My effect size of interest is the correlation (Pearson's $r$) between measurements from these new ...
1
vote
1answer
30 views

Which is the state of the art regarding methods for Association rule learning?

I have a set of data about which I would like as a first step, discover some interesting or frequent relationships. For this I know that there are several algorithms, like association or sequence ...
0
votes
0answers
24 views

ICC for estimating between-group variance in demographic groups?

This may be a silly or simple question, but it's bugging me. Say I have a US-based organization responding to a survey on workplace racial equality. In the organization, there are three racial ...
2
votes
1answer
90 views

Which ICC model to use for computer vs. human agreement

I have an agreement study with the following basic construction: ~ 400 items are rated by (1) a computer algorithm, and (2) a human rater. The ratings are on a continuous scale. I would like to ...
0
votes
1answer
253 views

Is high ICC essential when using HLM?

I have a question related to 'whether HLM is necessary or not.' My data has regions and firms with the time range of 2005 and 2010. I set each firm (with multiple observations within it) as the ...
0
votes
0answers
59 views

Concordance correlation coefficient or ICC

I have a set of observations. For each observation, I have six measurements. I would like to evaluate the test-retest reliability of a procedure which averages 3 of the 6 observations to produce an ...
0
votes
1answer
34 views

How to correlate time series to a variable

In an experiment, heart rate of participants is collected for 60 seconds (1 reading per seconds) and pupil diameter whilst performing one of three tasks. The success of the task is measured with a ...
1
vote
0answers
40 views

reliability of a device for my thesis and negative icc values in spss

first time here , hoping for a great answer! so my thesis is including using a sensor to measure how far is a a person leg is from the sensor. so i measured it with a ruler and compared with sensor ...
0
votes
1answer
99 views

ICC low, but still team variables in individual model?

I have data on two levels: team and individual. However, since the ICC on team level is low, I’ll not use a multilevel model but just a regular regression on individual level. After all, the impact of ...
0
votes
0answers
37 views

Unsure of which intraclass correlation model to use to compare consistency of measures across bouts

I am comparing heart rate (HR) values across repeated bouts of exercise. I have 20 participants who each performed 4 bouts of exercise at the same intensity. I want to test whether the HR values were ...
0
votes
0answers
65 views

Within-days and between-days repeat measures with mixed models

I tested participants in sports performance using a repeated measures design assessing the impact of two factors, position (2 levels) and speed (3 levels) on a number of dependent variables (modelled ...
1
vote
0answers
26 views

What test can I use for statistical significance?

This is a graph of the distributions of population proportion (y-axis) by population density (x-axis) for four different modal ages of housing stock. The black line, which we are interested in, ...
2
votes
0answers
27 views

Low ICC for Healthy but Large Difference from Diseased

I have a test I am interested in further investigating for diagnostic purposes. The test involves 6 trials, giving a mean result and variance for each person. What we have found through Mann-Whitney ...
1
vote
0answers
96 views

Which nlme model formula is correct for my data?

I want to fit a linear mixed model for my data, after reading around now I am slightly confused which one of the three models I had mentioned below is correct for my data. In my data I want to check ...
1
vote
1answer
46 views

Intraclass correlation with dependent sample units for inter-rater realibility

I have to calculate intraclass correlation to quantify intra and inter-reliability. However, I am not totally sure whether the classical two-way ICC is appropriate because of the correlation among ...
0
votes
2answers
40 views

Computation of correlation between variables pertaining to groups and variables pertaining to subjects included in groups

I have a number of observations that are organized in groups and subgroups. I would like to estimate the correlation coefficient between a variable that refers to each group and a variable that refers ...
1
vote
1answer
1k views

Does Cronbach's alpha reveal reliability information over and above the intraclass correlation coefficient?

I have seen in some papers that they have reported ICC as well as Cronbach's alpha. Does Cronbach's alpha provide more information about the reliability of a measured variable (e.g. heart rate at two ...
2
votes
1answer
437 views

Partitioning variance within a level in a 3-level mixed effects model

I have a 3-level repeated measures model setup observations (level-1) nested within patients (level-2) nested within providers (level-3). One of my questions is the percentage of variance explained at ...