Multiclassification: precision-recall from scratch vs sklearn

Question

I would like to know if there´s any issue behind using sklearn's precision/recall metric functions and coding up from scratch in a multiclass classification task. I noticed some researchers go by implementing this from scratch (multiclass) when it is clear such experience researcher cannot be unaware of sklearn's provided functions.

For example in this, a 5-class classification task. The research calculates precision and recall like so:

Pred = model.predict(Test_X, batch_size=32)
Pred_Label = np.argmax(Pred, axis=1)

ActualPositive = []
for i in range(NoClass):
    AA = np.where(Test_Y_ori == i)[0]
    ActualPositive.append(AA)

PredictedPositive = []
for i in range(NoClass):
    AA = np.where(Pred_Label == i)[0]
    PredictedPositive.append(AA)

TruePositive = []
FalsePositive = []
for i in range(NoClass):
    AA = []
    BB = []
    for j in PredictedPositive[i]:
        if Pred_Label[j] == Test_Y_ori[j]:
            AA.append(j)
        else:
            BB.append(j)
    TruePositive.append(AA)
    FalsePositive.append(BB)
Precision = []
Recall = []
for i in range(NoClass):
    Precision.append(len(TruePositive[i]) * 1./len(PredictedPositive[i]))
    Recall.append(len(TruePositive[i]) * 1./len(ActualPositive[i]))

When he could probably use:

sklearn.metrics.precision_score(y_true, y_pred,..)
sklearn.metrics.recall_score(y_true, y_pred,...)

But the researcher computes confusion matrix using scikit-learn API like so:

ConfusionM = confusion_matrix(list(Test_Y_ori), Pred_Label, labels=[0, 1, 2, 3, 4])

Answer 1

Sometimes it is much easier to implement directly. A situation whereby no functionality from sklearn is used, so it is too much a dependency to have large library just for confusion matrix.

If you would like to compute confusion matrix for multi-classifier, that is no more than a contingency table.

Let's use a synthetic data, 100 prediction and true classes, and compute confusion matrix with pandas cross-tabulation method with normalising over columns, i.e., diagonals of the resulting matrix will give recall but more accurate naming would be Top-1 score while recall/fall-out has only a connotation for binary classifiers. Here is the sample code:

import numpy as np
import pandas as pd
classes = ['a', 'b', 'c', 'd']
np.random.seed(42)
predicted_classes = np.random.choice(classes, 100)
true_classes = np.random.choice(classes, 100)
# recall: Top-1 scores
confusion_matrix_column_normalized = pd.crosstab(predicted_classes, true_classes, normalize='columns')
np.diag(confusion_matrix_column_normalized)
# Top-1 scores a = 0.15, b = 0.40, c=0.17, d=0.29

Please note that we use predicted_classes as first argument on pd.crosstab, while we want to group over predicted classes.

There are some discussions on how to form confusion matrix, see wikipedia discussion transposed confusion matrix, so depending on the field naming of recall and precision may reversed.