Tagged Questions
1
vote
0answers
196 views
Which correlation analysis is appropriate for binary and ordinal variables?
Violin players were asked to rate 5 violins from least to most rich on a [0 1], 0.05-increment scale, ...
2
votes
0answers
74 views
Testing graded response model thresholds for significance?
I have a data set in which two raters have each rated N samples using a 5-point ordinal rating scale. My primary interest is in whether these two raters make significantly different use of those ...
7
votes
1answer
215 views
What am I measuring when I apply a graded response model to the “Hunting of the Snark” dataset?
In another question, I asked about the statistical validity of StackExchange's "Hunting of the Snark" dataset, and whether or not we could draw any conclusions from its results. I measured a few ...
7
votes
1answer
670 views
How to perform inter-rater reliability with multiple raters, different raters per participant, and possible changes over time?
Participants were rated twice, with the 2 ratings separated by 3 years. For most participants the ratings were done by different raters, but for some (< 10%) the same rater performed both ratings. ...
3
votes
2answers
217 views
Combining ratings from multiple raters of different accuracy
I have a data set of J raters who each give ratings to I objects (on say a scale of 1 to 5). My goal would be to construct some kind of overall rating for each i based on all of the raters' scores.
...
3
votes
2answers
311 views
The best measure of reliability for interval data between 0 and 1 [closed]
I have 6 sets of interval data each of which between 0 and 1. Each set, calculated by a computer program, is related to the degree of similarity between some sounds (pairwise).
What do you think in ...
12
votes
2answers
7k views
Inter-rater reliability for ordinal or interval data
Which inter-rater reliability methods are most appropriate for ordinal or interval data?
I believe that "Joint probability of agreement" or "Kappa" are designed for nominal data. Whilst "Pearson" and ...
