I would like to understand the difference in what a test with Cohen's Kappa tells you about your data compared to what McNemar's test tells you.
If Cohen's Kappa measures agreement between two classifiers and McNemar measures how significant the difference between two classifiers is, then how do the two tests differ in the insight they provide? Since that sounds like more or less the same thing to me.
I wondered about this difference, as I applied both tests to my data but got very different results, while I expected the results to more or less match.
this is what I got for running McNemar (first figure) and Cohen's Kappa (second figure) on my data (comparing results from different classifiers on test data).
The cells in the first figure show the strictest significance level under which the two models of that cell are determined to be different. The cells in the second figure show the kappa value for the two models of that cell.