# Customized F1-Score for multi-class classification

Let's consider a multi-class classification problem with 4 classes: 0, 1, 2, and 3

F1-Score 'macro'-averaged is calculated like that:

F1_macro = 0.25*F1_class0 + 0.25*F1_class1 + 0.25*F1_class2 + 0.25*F1_class3


On the other hand, supposing that the relative supports for the classes are:

• class 0: 0.4
• class 1: 0.1
• class 2: 0.4
• class 3: 0.1

F1-Score 'weighted'-averaged is equivalent to:

F1_weighted = 0.4*F1_class0 + 0.1*F1_class1 + 0.4*F1_class2 + 0.1*F1_class3


Now my question is: does it make sense to consider a customized version of the scores where the weights' values follow an empirical rule? Let's say for example that I am interested in obtaining a model that predicts well minority classes, and in particular class 3. Could the following metric be a valid approach?

F1_custom = 0.1*F1_class0 + 0.35*F1_class1 + 0.1*F1_class2 + 0.45*F1_class3


I understand that considering the weights as inversely proportional to the relative support could result in overproportional weights in case of severe imbalance, but considering them like that should not be a problem in this sense.

I am aware that there are better metrics, that are also threshold-independent (e.g. AUPRC or AP) that can be extended for a multi-class problem and for which I could also apply this method, but for the moment I am interested in validating this 'custom averaging' method, so I would like to understand if it is feasible for a simple metric like F1-Score and then extend it to other metrics (like AP).

In principle, there is no problem with defining your custom scores. This can be a custom f1-score that you care about, a customized score obtained from confusion matrix, or some other arbitrary formula. The important thing is, it should make sense regarding your problem.

Let's say for example that I am interested in obtaining a model that predicts well minority classes, and in particular class 3. Could the following metric be a valid approach?

F1_custom = 0.1*F1_class0 + 0.45*F1_class1 + 0.1*F1_class2 + 0.35*F1_class3

Regarding this, I'd expect you give more weight to the F1 score of class 3.

Maybe you can find more people doing customizations by looking for custom losses defined on confusion matrices. People usually assign a cost of misclassification for each class pair, i.e. $$C_{ij}$$, and multiply the costs with the entries in the confusion matrix.