I am having some trouble trying to decide what measure of inter-rater reliability to use in a study. Part of a larger study involves accurately determining when participants began (onset) and stopped (offset) writing. I have video footage which captures the writing, and can determine the time of onset/offset to the nearest 0.01 second. The time is taken from a stopwatch which was running continuously from the start of each experiment, with multiple onset/offsets in each experiment. The onset/offset have been defined, and I would like to determine the inter-rater reliability for 2 raters. The data looks like this:
Participant A trial one - onset: 21.35 offset: 24.55 trial two - onset: 54.50 offset 57.75 Participant B trial one - onset 10.35 offset 21.15 trial two - onset 35.65 offset 39.90
and so on and so forth. Two raters would have data like the above and I would like to calculate if they are reliably similar. I looked at Kappa statistic, Krippendorff’s alpha, intra-class correlation, etc., but none seems appropriate for continuous time measurements.