I'm trying to describe inter-rater reliability for a group of 5 researchers working on the same project. We're administering the Hamilton Anxiety rating scale which has 14 items, each item having 4 ordinal levels. Each rater has watched the same video of the same subject being tested and rated it independently. There is only one subject. So we have 5 columns with 14 rows of ordinal data. There is another row that has total HAMA scores for each rater.
To my mind this could be interpreted by simply looking at the data and seeing how much agreement there is. However, some kind of inter-rater statistic has been requested. Does anyone have any recommendations for this? Is something like the Kendall coefficient of concordance or Krippendorff's alpha work here with only one subject?
EDIT: I've noticed that when running something like % agreement in the 'irr' package, it will produce results for '5 raters and 14 subjects'. But of course I have one subject and 14 items.
Cheers for the help