I am calculating inter-rater agreement between two scales. If I have two scales which measure the same concept but are slightly different in length (the technician questionnaire is a few questions longer than the patient questionnaire) do I need to standardise the scores first? If yes, would you just z-score it? And, which inter-rater agreement calculation would you use?
1 Answer
$\begingroup$
$\endgroup$
This doesn't completely answer my question but just in case some other poor fellow is looking for answers to something similar. This paper - https://lra.le.ac.uk/bitstream/2381/3915/3/Comparing%20Rating%20Scales%20of%20Different%20Lengths.pdf talks about comparing rating scales of different answer lengths and the merits of different types of data transformations.