2
$\begingroup$

I'm working on a classification problem with a very imbalanced dataset. Many papers mention a "weighted cross-entropy loss function" or "focal loss with balancing weights". I can't find any of those in tensorflow (tf.keras to be precise) but there is a class_weight parameter in model.fit(). Is there a difference between those two things or is this just the way tensorflow implements weighted loss functions?

$\endgroup$
1
1
$\begingroup$

The weighted cross-entropy and focal loss are not the same. By setting the class_weight parameter, misclassification errors w.r.t. the less frequent classes can be up-weighted in the cross-entropy loss. The focal loss is a different loss function, its implementation is available in tensorflow-addons.

$\endgroup$
2
  • 1
    $\begingroup$ Sorry, that maybe was a bit misleading, I know that those are different functions. My question is if the class weight parameter is just the way of passing weights to the loss function in tf.keras or if there are also special loss functions that directly take weights. And if the second case was true, wat would be the difference of those options. $\endgroup$ Dec 27 '19 at 23:11
  • $\begingroup$ Yes that's correct, in Keras you use the class_weight parameter, which under the hood gets converted to per-sample weights, depending on class_weight, which are used to compute a weighted loss across all samples. $\endgroup$
    – sebp
    Dec 28 '19 at 16:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.