I want to measure the inter-agreement rate between two physicians that rated the same set of images with two different methods (rater1 used method1, rater2 used method2) BUT
Rater1 rated the the images with method1 : 0 (absence of the lesion) or 1 (presence of the lesion)
Rater2 rated the images with method2 : 0 (absence of the lesion), 1 (small lesion), 2 (little bigger lesion), 3 (intermediate size of lesion), 4 (big lesion) or 5 (very big lesion)
Like you can see my rating variables have different set of catergories (2 categories for rater1 and 6 for rater2). how can I deal with that? is there a way to change the scores 1,2,3,4 and 5 (of rater2) to 1 (so we find our self in the classic setting 2 categories vs 2 categories) but still give some sort of weights to them (because if rater1 with method 1 scored 0 when the rater2 with method 2 scored 5 it's more serious than if rater1 scored 0 when rater2 scored 2 for example)
I want to know if method 1 is comparable to method 2 based on this scores (i.e. method 2 is the gold standard). Basically how method 1 perform compared to method 2 based on scoring variables/ how is method 1 reliable compared to the gold standard method 2?
PS: if you have another way then KAPPA to answer the latter question I take it !