I was presented with a requirement of comparing two sources of diagnoses on the same subjects, resulting in two 4-level categorical variables. I offered to calculate Cohen's Kappa, but the interested party would prefer a p-value as a result.
a) Is there a method that natively does this? b) Does it make sense to do one McNemar test for each of the four variable levels, making binary variables like 1 and non-1, then 2 and non-2 etc. ?