In semantic segmentation and similar applications, sparse categorical cross entropy is often used as a loss function. Now it usually happens that samples are imbalanced. In my case, I have one class that accounts for about 60% of the total image areas and the remaining 30 classes share the rest.

As shown in this tutorial, tf.keras.losses.SparseCategoricalCrossentropy takes not only X and y (images and masks/labels) but also weights. Now I wondered how to find the most suitable weights?

So far, I just searched through all the masks in the training set and created an array with the total pixel counts for each class. Then I just calculated weights $w_i$ by the inverse pixel count as $w_i = \frac{\text{max}(\text{count})}{\text{count}_i}$ for each class.

Let's say we have images with $n$ unequally frequent classes $c$ and a model that simply classifies every encountered pixel as class $c_i$. Then I would like to have weights that make this behaviour equally costly no matter the class $c_i$; i.e., $\text{loss}_i = x\; \forall\; i \in \{1,...,n\}$.

Does this make sense? And is it possible to implement?

On the internet I found a formula for catrgorical cross entropy (I am not sure if it is the same for sparse categorical cross entropy, though):

$\text{cce} = - \frac{1}{N} \sum\limits_{i=1}^N\sum\limits_{c=1}^C1_{y_i}\in C_c \log{p_\text{model}}\left[y_i \in C_c \right]$

Maybe someone has the mathematical knowledge to find a solution to this problem (or to tell me where I am wrong).

  • $\begingroup$ Why is the class imbalance a problem for your work? Statisticians tend not to see class imbalance as such an issue. $\endgroup$
    – Dave
    Commented Jul 2, 2021 at 14:45
  • $\begingroup$ @Dave It is an issue because the model is supposed to reduce the loss function. Here, it is highlighted what the problems with accuracy are, when you have unbalanced classes. A non-weighted categorical cross entropy loss function will similarly lead to a model that only predicts the most common class. It's the cheapest solution but it makes the predictions completely useless. $\endgroup$ Commented Jul 2, 2021 at 15:00
  • $\begingroup$ Accuracy is a problem when there is perfect class balance, too. $\endgroup$
    – Dave
    Commented Jul 2, 2021 at 15:08


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.