I have essay scores from one experiment. I have two raters and each rater separately graded the essays by using rubrics. The rubrics consist of 5 criteria, rating scales (1-5) and "descriptions"(clearly definition of a certain level on that criterion) in each cell.
I have three questions: - Are scores of each criterion regarded as ordinal or interval data? Which statistical method is more appropriate for rater agreements? - Should I calculate rater agreements for the scores per criterion or the total score (adding up scores of all criteria)? - Can I run parametric analysis to compare the essay scores of different groups of students?