The Intra-class Correlation Coefficient measures the degree to which a categorical variable accounts for variability in a continuous variable. It is often used to index the degree of clustering in a dataset.

learn more… | top users | synonyms (1)

0
votes
0answers
15 views

type of ICC coefficient to use in my study for inter und intrarater-reliability

I am new to the subject and not sure, which ICC I have to use in order to analyse inter/intrarater-reliability of my measurements. A physician examined 649 healthy individuals with an ...
0
votes
0answers
6 views

Comparing TrueScore with the System Score

I have collected three human raters scores for an essay question(N=54). I am building Automated Scoring Algorithem. So I need to compare my machine score with the Human rater score. All are using 0-10 ...
2
votes
1answer
46 views

Multilevel logistic regression with a random slope(s)

I would like to specify the two-level logistic regression model with random intercept and random slope. Dependent variable: hospitalization (1) or no-hospitalization (0). Independent variables: age, ...
0
votes
0answers
19 views

Multilevel analysis. Between hospitals variance

Please help me with the multilevel analysis. Certain medical procedure is performed/not performed in hospitalized patients. The % of patients that receive this procedure varies across different ...
1
vote
0answers
34 views

Intraclass Correlation Coefficient in mixed model with random slopes

I have the following model m_plot fitted with lme4::lmer with crossed random effects for participants (...
1
vote
0answers
82 views

How to interpret glmer results (variance, correlation and ICC)

I'm a beginner in statistics and I have to run multilevel logistic regressions. I am confused with the results as they differ from logistic regression with just one level. I don't know how to ...
0
votes
0answers
13 views

ICC to assess independence of observations in a cluster randomized trial

How can I compute an ICC to assess independence of observation assumption using SPSS for ANOVA. The data on two DVs were collected from participants one time over 28 weeks in 28 clusters (so this is ...
1
vote
0answers
28 views

What are the tests to measure agreement between two methods of measurement when the unit of measurement is different between the two methods?

Bland-Altman plot measure the agreement between two different methods which measures the same variable. As far as I understand Bland-Altman can measure only if the two methods have the same unit of ...
1
vote
0answers
25 views

Intraclass Correlation Coefficient Question

I have two datasets in which I probably need to use the intraclass correlation coefficient, and I want to make sure I choose the right type. Dataset 1: I had 20 randomly selected human subjects rate ...
0
votes
1answer
54 views

Classification using correlation

Given two correlation matrix (each $p \times p$), where each belongs to a different group, is it possible to classify a new sample into one of the group (based on the correlation matrix only)? What ...
2
votes
1answer
47 views

How to test whether ICCs from different groups are significantly different?

I need to compare two generalizability (G) coefficients for data that are from two separate populations. G coefficients are a type of intraclass correlation coefficient (ICC). The literature on ...
0
votes
0answers
50 views

Should I use both paired t-test and intraclass correlation coefficient to examine the interrater reliability of an instrument?

As title, I am assessing the interrater reliability of a questionnaire (actually a Barthel Index (BI)) performed by four assessors (A,B,C,D) on about 25 test subjects. I plan to examine the IRR from ...
0
votes
0answers
16 views

Interpreting the relInterIntra {irr} results in R

I am using this for a reliability analysis test for quantitative values between two raters. I got my results using the "relInterIntra {irr}" package in R, but the reported values gave me several ...
0
votes
0answers
19 views

Non-normal data (ceiling effect) and intraclass correlation

I am wondering if there is any way of doing an intraclass correlation, two-way, mixed, with non-normal data (in this case a ceiling effect)? Alternatively, what the interpretation (if any) would be? ...
0
votes
0answers
50 views

'irr' R Package giving ICC values outside of 0 - 1

I ran a study looking at MEG (EEG with magnets) responses in the gamma-band (30-50Hz) range across multiple days. The following data is the timing of the initial response in that range (in seconds), ...
0
votes
0answers
22 views

How to analyse individual consistency from three non-normally distributed variables?

I would like to know if individuals were consistent across three different days. I measured the latency of individuals for a maximum of ten minutes. Many individuals had a latency of 0 and some ...
0
votes
0answers
22 views

Computing the Correlation with Skewed Repeated Measures Data

Suppose I have two variables (e.g., Blood Pressure and Cholesterol) and I want to investigate their association across time. Also suppose that each individual has been measured three times ...
0
votes
0answers
16 views

Reliability Measure for Unequal Subjects between Raters?

I am comparing two medical imaging analysis techniques, and I would like to assess the inter- and intra-rater reliability for using each method. I am planning the study design, and would like help ...
0
votes
1answer
294 views

Can ICC values be negative?

I am calculating the ICC as per the method of Shrout-Fleiss (1979), using the (3, 1) model in which the judges are fixed as are the targets. Some of my ICC values are negative. Is this to be ...
0
votes
0answers
36 views

Intraclass correlation coefficient: mean of raters, or better rater?

Suppose I have a set of stimulus, that was rated by three independent rater. I then estimated the Intraclass correlation coefficient, with a resulting value of 0.881. Let's consider a practical ...
0
votes
0answers
25 views

correlate/compare two scales measuring different constructs

I need to compare a multidimensional pain scale item response >3 pain is present and a scale measuring emotional suffering with classified levels of suffering (High, intermediate and low). Will ICC ...
2
votes
0answers
55 views

Likert scale inter rater agreement

What is the best inter rater agreement test for Likert scale type questions. As far I saw that cronbach alpha is for internal consistency, or it shows how good items are related to describe the main ...
1
vote
0answers
56 views

Correlated predictors in linear model: where does shared variance go?

I analyze a dataset consisting of responses to two conditions with a multiple regression analysis. I dummy code the two conditions to retrieve the contribution of each of them to the data. Up to here, ...
0
votes
0answers
49 views

Measure of accuracy or measure of agreement for continuous data with true values

I have two continuous variables. Variable $X$ represents the true values (basically it is atmospheric pressure measured via a barometer). Variable $Y$ represents a way to approximate the atmospheric ...
0
votes
0answers
79 views

Intraclass Correlation Coefficient Confusion

Many times in statistics we are using a statistic in different ways. I think Intraclass correlation coefficient is one of them... Intraclass Correlation Coefficient can be a A) measure of inter-rater ...
0
votes
0answers
6 views

Assessing test-retest reliability of a questionnaire [duplicate]

I am testing the test-reliability of a newly developed questionnaire over two time points. The study involves parents answering about their child over the two occasions. Would intraclass correlation ...
2
votes
1answer
31 views

Statistic to find out association between ordinal data

I have a 4x3 contigency table with ordinal data for a research finding in tuberculosis, I analysed cytology patterns and afb patterns; cytological pattern $A,B,C,D$ (independent variable) and afb ...
0
votes
0answers
54 views

What is a “subject” in ICC?

I have been working in R and have finally got some code working regarding intraclass correlation coefficients. In the output I get something called "subjects". I thought it was to do with the number ...
0
votes
0answers
86 views

Mean pairwise Pearson correlation

Consider a set of $M$ multidimensional observations $X=\{x_i\}_{i=1..M}$ for $x_i\in\mathbb{R}^N$, and define $c_{ij}$ to be the Pearson correlation coefficient between $x_i$ and $x_j$. Is there a ...
0
votes
1answer
36 views

Comparison between fat-matrix

I have two data sets obtained from two different kind of chemical analysis: GC-MS and IR. Both the chemical analysis were performed using the same samples. My aim is to know if there are some ...
2
votes
1answer
242 views

ICC in a linear mixed model - longnitudial data - why is this the correlation between time periods?

I'm confused by the description of the intraclass correlation (ICC) for a linear mixed model with longitudinal data from this material. A screen capture is shown below. What I'm confused about is ...
0
votes
0answers
77 views

Intraclass correlation (ICC) per item instead of across items

I am using the intraclass correlation coefficiant to determine inter-rater reliability for a set of pictures that were rated by several raters on a scale from 1-9. I am using the two-way mixed model ...
2
votes
1answer
136 views

Measuring intra-observer correlation

We're far from what you'd call expert statisticians, so we'd like to ask some help in an ongoing medical research. We have 60 subjects. A researcher took at least 3 times the measurements of 8 of ...
2
votes
1answer
441 views

How to compare repeatability (ICC) of different groups?

I have calculated the ICC values for two groups and would now like to compare the ICC values to determine if the groups differ in their repeatability. In the literature people have simply used t-tests ...
1
vote
0answers
264 views

Intra- and inter- observer error, and the intra-class correlation coefficient (ICC)

I have 3 observers, who each perform multiple observations in which they count a phenomenon. I'm trying to quantify both inter- and intra- observer error for this study. I'm new to this and am finding ...
2
votes
1answer
107 views

ICC - unclear comment from reviewer

I'm new here and not a statistician, so I apologize in advance and I sure hope someone can help! I'm working on a study where we have 10 locations $(5 \ intervention, 5 \ control)$. We have two ...
1
vote
1answer
159 views

Low intraclass correlation although between variance is very high in random effects model

I don't understand why a random effects model can give you a very low rho (intraclass correlation coefficient), if the between variance (between subjects) in your ...
1
vote
0answers
234 views

How to analyze intra-rater reliability with 31 raters who all rated 4 subjects twice

I'm trying to analyze the intra-rater reliability of an occupational therapy assessment. The assessment consists of 57 items, where some items are ratio data and some items are ordinal data. In my ...
1
vote
0answers
50 views

Averaging correlation matrices among subjects

Suppose I have N correlation coefficient matrices, containing the Pearson correlation coefficients of k different time series of ...
2
votes
0answers
29 views

Combining critic scores into single rating

I have a collection of scores from critics for various restaurants. Some restaurants are rated by multiple critics and some by only a single critic. Each critic rates restaurants using different ...
2
votes
1answer
117 views

Agreement of clustered data

I have the following situation: I have analyzed several data curves from a group of patients (16 curves per patient) with different analysis methods and want to test for the agreement of the methods. ...
3
votes
1answer
364 views

how to partition the variance explained at group level and individual level

Based on results of Null model, some scholars suggest variance attributed to group level and individual level is computed using variance component output, whereas other suggest going the standard ...
2
votes
1answer
1k views

ICC and Kappa totally disagree

Much has been written on the ICC and Kappa, but there seems to be disagreement on the best measures to consider. My purpose is to identify some measure which shows whether there was agreement between ...
2
votes
0answers
138 views

Comparing two dependent intra-class correlations (ICCs)

We are measuring two waves from a biological signal. As an example consider the ECG $P$ wave and $T$ wave. From a single trace, the amplitude of both of these waves is measured. We want to compare ...
7
votes
1answer
273 views

Intraclass correlation in the context of linear mixed-effects model

Suppose that one investigates the two sources of variability with data $y_{ij}$ acquired from $j$th subject under $i$th session ($i=1,2; j=1,2..., n$). A linear mixed-effects model can be formulated ...
1
vote
0answers
141 views

intra-rater reliability and assessment error - which ICC type to use? singular rater

I'm trying to figure out which ICC type to use to calculate MY measurement error using a particular assessment tool: background and data: PhD feasibility phase only. 10 pairs of videos of people ...
2
votes
0answers
108 views

Validity of grand mean intra-rater Kappa or ICC

For 5 raters, I have calculated intra-rater Cohen's Kappa statistics for the test-retest nominal ratings, intra-rater Kendall's tau-b statistics for the test-retest ordinal ratings, and intra-rater ...
2
votes
1answer
68 views

Looking for simple examples of how to calculate type III and type IV SS

I have data collected on five species of fish at half a dozen locations in a lake over four years. The categories are not (at all) fully crossed, and I have a lot of empty cells due to logistical ...
12
votes
2answers
610 views

Looking for Intuition Regarding Fixed/Random Effect Conundrum

I understand that we use random effects (or mixed effects) models when we believe that some model parameter(s) vary randomly across some grouping factor. I have a desire to fit a model where the ...
0
votes
2answers
601 views

What statistics should I use for evaluating the accuracy of predictions?

I have two variables representing 1) players' predicted fantasy football points and 2) players' actual fantasy football points scored. What statistics are best for assessing the accuracy of the ...