# How to compare 2 classifers using confusion matrix?

How to compare 2 classifiers using Confusion Matrix?

For example, if we have 2 confusion matrix(binary classification) obtained from different classifiers or using different features, how I can compare the performance of classifiers using Confusion Matrix?

confusion matrix
60 34
1  12

confusion matrix
90 4
8  5


As @Enrique mentioned, there are many statistics you can calculate from a confusion matrix. To determine which ones are appropriate depends on the specific characteristics of your problem, such as the relative costs associated with true positives and false positives.

Chapter 11 of Applied Predictive Modeling gives a very detailed overview of how to think about evaluating classification models.

An Introduction to Statistical Learning, which is freely available as a pdf, provides a less detailed overview in chapter 4.

• What statistics do you recomend to classification problem with unbalanced data? Aug 17, 2015 at 17:06
• If you have highly imbalanced data, a lot of the basic measures of performance won't be terribly useful (e.g., you can get a high accuracy by just predicting all negatives). AUC (area under the ROC curve) can be good because it can help you identify what cutoff to choose. That'll let you decide what ratio of true positives to false positives you're comfortable with. Positive Predictive Value is another option. Aug 17, 2015 at 17:12

From the confusion matrices you can compute the sensitivity, specificity, accuracy, precision, among other performance metrics for each of the classifiers. Then you can evaluate them in terms of those metrics. Here you can find the definition of several metrics and how they can be computed.

Confusion Matrix is a square matrix, which in the ideal case, its main diagonal must be valued and other sides must be none.

Confusion Matrix in binomial class:
------+-------
| TP  |  FP  |
--------------
| FN  |  TN  |
------+-------


Thus, there are two main methods to evaluate a predictor: Using accuracy or F1-score (in imbalanced cases should be used).

ACC = (TP + TN) / (P + N)

F1 = 2TP / (2TP + FP + FN)

So in your case, the second predictor is better with 89% accuracy than the first one with 67%.

Here is a Confusion Matrix Online Calculator.