# What is the best criterion for performance evaluation in classification algorithms?

Here are some results for ANN and KNN on abalone data set using Weka:

Result for ANN
Correctly Classified Instances  3183 76.203 %
Incorrectly Classified Instances 994 23.797 %
Mean absolute error      0.214
Root mean squared error  0.3349
Relative absolute error 58.6486 %

Result for KNN
Correctly Classified Instances  3211 76.8734 %
Incorrectly Classified Instances 966 23.1266 %
Mean absolute error      0.2142
Root mean squared error  0.3361
Relative absolute error 58.7113 %


KNN has high accuracy but ANN has low errors. So which of the two algorithms should I say is better?

Which is the more preferable criterion, accuracy or error?

What I understood was that error should decrease with high accuracy, but the results here are opposite. Why is this so?

• are you talking about classification or regression? – Simone May 30 '13 at 9:08
• Classification mainly – Haya Hallian May 30 '13 at 9:33

The methods appear to be very similar in performance and that's perhaps the main story.

But I don't think we can say much to help you decide.

Which is more important practically, making the right classification or reducing error? It can easily happen with mean error measures that even one odd observation pushes them up a fair bit.

I don't think that scrutinizing the figures of merit is going to be most informative here. Rather, focus on which observations were classified differently and think about what that implies. I don't know your software to advise how to do this. If it's not easy, you need better software.

Also, always use visualizations to see the classification in some space.

In this case, KNN is better than ANN. You should look at the accuracies, i.e., "Correctly Classified Instances".

Your logic about error and accuracy is correct. But here "relative absolute error" is probably different than what you imagined. See: http://list.waikato.ac.nz/pipermail/wekalist/2004-August/029217.html