I am testing different multi-class (16 classes) classification models (ANN, DNN, DL, and so on) and the overall accuracy varies from 0.75 to 0.92. Despite that 0.92 is greater than 0.75, is there a way to compare the outputs statistically? I do not know if cotigency tables are the way to go, since I want to compare the output with to the original label as well. For instance, in the following example:

Observed value Prediction 1 Prediction 2
A A   C
B B   B
A A   C
B B   B
B B   B
C C   C
B C   C
C C   A

How can I stablish if Prediction 1 is different than Observed value, or Prediction 2 different than Observed value or Prediction 1? What is the suitable statistical test for this purpose?

  • $\begingroup$ I answered a similar post and recommended not performing a lot statistical analyses on classifier results here: stats.stackexchange.com/questions/637282/… $\endgroup$
    – wjktrs
    Jan 20 at 6:08
  • $\begingroup$ Thank you, it was very enlightening $\endgroup$
    – aldo_tapia
    Jan 22 at 12:55


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.