1
$\begingroup$

Consider a classification setting like a medical test: Not finding an existing health issue might be much worse (by a factor of 50) than assuming an issue when there is none. I.e. a setting in which false negatives are much worse than false positives or vice versa.

In classification, cross entropy is a very popular loss function with a solid theoretical foundation. However, it does not consider different cost incurred by making a specific type of error.

According to this answer, simply adding a coefficient does not work as intended.

Do you know loss functions considering different severities of error types? I have not found any reference to such a function in my search.

I am aware of the fact that I could use class balancing and sampling techniques. However, I am not excited by the resulting model performance. When I used a different loss function however, the results were much better. The intuition behind the linked loss function is to use the f1-score, which is typically used as a metric as a loss function instead. Because the original metric does not possess the necessary mathematical properties of a loss functions, it is modified in way so that it does. I then modified the linked "f1-loss" to this

Generalized F-score,

in order to be able to penalize false negatives using the parameter beta (when recall is given a higher weight, then the FN-rate will be lower).

Even though the results are much better (compared to class balancing), I would like to use a loss function which is not just a repurposed metric. Ideally, I hope to use a loss function which is proper in the sense that it accounts for the different costs and is theoretically justified.

The equivalent problem in a regression setting might be stated as different costs for over- or underestimating the target variable.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.