I am trying to figure out what would be the best statistic to use to quantify the amount of agreement that exists between subjects who were asked to press a button whenever they felt a certain emotion while listening to a short (2min) piece of music. Plotted as time series, the raw data looks something like this (one subject per line; each event(keypress) represented by a blue dot):
I initially thought of Kendall's tau, but I believe that is only suited to rank data, which is not what I have here. I guess Kendall's W or Cronbach's alpha would be appropriate, although I am not sure whether these are robust to my data having unequal numbers of total events that the different subjects define, e.g. in the screenshot, the first subject defined 3 events, the second subject only one event, etc.
I think correlation would be unsuitable here, as I want to obtain the agreement between all subjects as opposed to between pairs of subjects.
I am also unsure whether what I need to quantify is in fact inter-rater agreement vs inter-rater reliability (e.g. see Liao et al 2010).
What I've described so far aims to obtain a single number that expresses the overall agreement between subjects across the entire span of the musical piece. However, the piece lends itself to being divided on the time axis into bins/epochs (of varying widths), on the basis of music theory. An additional (and perhaps more meaningful) measure would thus also be to compute the inter-rater/subject agreement/reliability separately for each bin/epoch. Would the employed statistic (e.g. Kendall's W) change in that case?