The intraclass-correlation tag has no wiki summary.
0
votes
2answers
159 views
What statistics should I use for evaluating the accuracy of predictions?
I have two variables representing 1) players' predicted fantasy football points and 2) players' actual fantasy football points scored. What statistics are best for assessing the accuracy of the ...
1
vote
0answers
206 views
Computing Cronbach's Alpha and Intraclass Correlation coefficient from an lmer model
I am trying to calculate the Intraclass Correlation for a rater study using R and the library lme4 and the function lmer. The data has the following design: The same 6 raters (at least 4) are rating ...
1
vote
2answers
128 views
Several intraclass correlations (ICC) when modeling heterogeneous variances?
In a nested design (e.g., students in schools, ants in colonies, individuals in a species, leaves in a tree), ICC is defined as the proportion of variance among observations from the same subject ...
2
votes
0answers
50 views
Assessing the ability of two quantitative methods measuring the same thing
Suppose I have two scientific approaches to measuring the same physical quantity taking values between $[0,1]$. One (say method A) is generally accepted to be more accurate under certain conditions, ...
3
votes
0answers
623 views
For intraclass correlation coefficients (ICC), interpret both “single” and “average” measured ICC values for reproducibility test?
Say there's a new machine for use in testing a biochemical marker's concentration in the blood, and we would like to know the reproducibility of this test.
We draw blood samples from 10 volunteers, ...
0
votes
0answers
105 views
Intraclass correlation coefficient, non parametric data
Does the ICC require that my data are normally distributed?
If more information about my data is needed, just ask.
1
vote
0answers
81 views
Intrarater reliability
I would like to estimate the intrarater reliability in a MRI study. Different regions of interest were manually traced with a mouse-driven cursor in order to estimate their volumes. All data were ...
3
votes
2answers
174 views
Inconsistency of findings between Kappa and ICC for IRR study
I used the following R script to calculate the Kappa and ICC for a measured done by two raters on 23 subjects:
...
5
votes
0answers
143 views
What GEE-exchangeable method can do that robust variance can't?
I asked a related question before here on the difference between GEE method with exchangeable varcov structure v. Robust standard errors known as Huber White method in group randomized trials. As ...
5
votes
1answer
217 views
How to statistically test whether there is any nesting effect in the data?
I have a sample including all students within 52 schools which have been randomly assigned to be treatment or control (we have group randomized trial in which all students in each school is either ...
7
votes
1answer
639 views
How to perform inter-rater reliability with multiple raters, different raters per participant, and possible changes over time?
Participants were rated twice, with the 2 ratings separated by 3 years. For most participants the ratings were done by different raters, but for some (< 10%) the same rater performed both ratings. ...
0
votes
0answers
207 views
Nonparametric ICC
Anyone can suggest me a nonparametric version of the intraclass correlation coefficient (ICC)? My model have the following details: two-way random effects model, consistency, average ratings as unit. ...
1
vote
0answers
82 views
Intraclass correlation to validate a measure
I would like to understand whether the output of an algorithm I developed agrees with the ratings of external observers. The number of observers is 10 and each of them rated on 2 3-point-Likert items ...
1
vote
0answers
102 views
Within group correlation of pearson's residuals
I have social network data in which an "ego" names a friend "alter". I am running a regression in R in which attributes of alter are predictors of outcomes for ego. So each observation is dyadic with ...
1
vote
1answer
955 views
Justification of aggregation in multi-level model - ICC in SPSS
I am running a multi-level model on data which is set out with one row per individual survey return, each with Individual ID and Organisational ID as columns. There are variables which can be ...
1
vote
0answers
90 views
How to compute effect size for different measures of association?
My colleagues and I are looking for guidance on how we might compute a common effect size (ES) from the following measures of association:
Pearson's r
Spearman's rho
Kappa
ICC / intraclass ...
1
vote
0answers
410 views
How would one calculate intrarater reliability in Stata?
My question follows from a previous excellent response in another post:
Intra- and inter-rater reliability on the same data
Wow, what a truly great response! I am running through a similar problem ...
0
votes
1answer
87 views
Correlated errors between variables [between / within subjects, within variables, everything]!
I am trying to figure out good model to analyze social dynamics using a model. I have commenting data from a website. Such as 500 articles, 10000 users, and their 75000 comments. Every user does not ...
2
votes
1answer
286 views
ICC reduced to 0 in a multi-level model. Should this be suspicious?
I calculated a sequence of multi-level models with stata (xtmixed). I have a sample of 800 kids in 46 classes and I want to explain their individual level of prejudice by the proportion of fully ...
1
vote
1answer
210 views
Assumptions for intraclass correlation
I am having difficulty finding information on the assumptions for the intraclass correlation. Can someone please tell me what they are?
13
votes
1answer
667 views
Intraclass correlation (ICC) for an interaction?
Suppose I have some measurement for each subject at each site. Two variables, subject and site, are of interest in terms of computing intraclass correlation (ICC) values. Typically I would use ...
6
votes
0answers
427 views
Intraclass Correlation Coefficients (ICC) with Multiple Variables
Suppose I have measured some variable in siblings, which are nested within families. The data structure looks like this:
family sibling value
------ ------- -----
1 1 y_11
1 2 ...
4
votes
1answer
2k views
Intra- and inter-rater reliability on the same data
I've read some of the answers here (specifically the 'Inter-rater reliability for ordinal or interval data' one). I'm still somewhat perplexed though!
I have data for 4 raters, who each rated every ...
6
votes
2answers
693 views
Comparing correlation coefficients
I have two sets of data where I have at ~250.000 values for 78 and 35 samples.
Some of the samples are members of a family and this may have an effect of the data. I have calculated pairwise ...
3
votes
2answers
168 views
Specification of a random slopes model with two grouping factors
Suppose people take a 10-item exam. For each item, $k = 1,2,…,10$, exactly one rater assigns a score to exactly one person, with the constraint that no person sees the same rater twice. There are $i = ...
4
votes
1answer
294 views
Assessing independence of observations using intraclass correlation when some groups have small group sample sizes
Context
I was talking to a researcher in the following situation.
Participants (n = 500) were sampled from schools.
Participants came from around 50 different schools.
The number of participants ...
5
votes
1answer
3k views
Intraclass correlation coefficient vs. F-test (one-way ANOVA)?
I'm a little confused regarding the intraclass correlation coefficient and one-way ANOVA. As I understand it, both tell you how similar observations within a group are, relative to observations in ...
2
votes
1answer
568 views
Assumption of additivity for intra-class correlation
My question concerns the assumption of additivity for intraclass correlation. I shall first explain what I have done and then end with my questions.
I want to calculate inter-rater reliability using ...
3
votes
1answer
6k views
Inter-rater reliability using Intra-class correlation with ratings for multiple objects on multiple properties
Initial Question:
I am trying to calculate inter-rater reliability. Previous researchers in this area have used intraclass correlation. SPSS has options for two-way random, mixed random and one-way ...
7
votes
2answers
798 views
Intraclass correlation and aggregation
Imagine that:
You have a sample of 1000 teams each with 10 members.
You measured team functioning by asking each team member how well they think their team is functioning using a reliable multi-item ...
