0
$\begingroup$

I am currently working on implementing a weighted binary crossentropy loss function as described in the U-Net paper

def weighted_pixelwise_crossentropy(self, wmap):

    def loss(y_true, y_pred):

         return losses.binary_crossentropy(y_true, y_pred) * wmap

    return loss

Same as in the paper, wmap contains both a weight map for class balancing as well as a weight map to highlight object borders. In the images I use for training, for 1 foreground pixel there are roughly 30 background pixels, so I would have assumed using a weight map should have improved the training and validation accuracy. But I get roughly the same results as when not using any weight map at all. Does anybody have any potential ideas why this might be the case?

$\endgroup$
2
  • $\begingroup$ surely if the classes are as unbalanced as you say, mean IoU makes more sense than accuracy as a metric $\endgroup$
    – shimao
    Commented Feb 26, 2019 at 16:37
  • $\begingroup$ Sorry, I should have been more precise. With accuracy I in fact mean the F1 score (or dice coefficient), which seems not be influenced by using a weight map. $\endgroup$ Commented Feb 26, 2019 at 16:49

1 Answer 1

0
$\begingroup$

It seems you are applying the same wmap to all images in the training dataset. This is most likely not what you want; rather, you probably have a wmap for each labelled image. If so, the approach of a higher-order function, ie a function returning a loss function will not suffice.

Instead, consider embedding the wmap as a separate channel in the labelled masks. Ie, your y_true would consist of two channels: A binary mask and a wmap:

...while producing your dataset Ys
y_true = ...load labelled mask
weight_map = ...calculate weight map for y_true

# stack mask and wmap as separate channels to produce shape (H,W,2)
y_true_weights = np.dstack((y_true, weight_map))

...Which will also guarantee image augmentations affect both mask and wmap equally.

Then, your custom loss function could be changed to:

import tensorflow as tf
from keras.losses import binary_crossentropy
from keras import backend

def binary_crossentropy_weighted_loss(y_true_weights, y_pred):
    # split ground truth and weight map channels, each now of shape (H,W,1)
    y_true, weight_map = tf.split(y_true_weights, num_or_size_splits=2, axis=-1)

    # calculate cross-entropy same way as keras.losses.binary_crossentropy
    y_pred = tf.convert_to_tensor(y_pred)
    y_true = tf.cast(y_true, y_pred.dtype)
    cross_entropy = backend.binary_crossentropy(y_true, y_pred, from_logits=False)

    # apply weight map
    cross_entropy_weighted = tf.multiply(cross_entropy, weight_map)

    # calculate mean ce same way as keras.losses.binary_crossentropy
    return backend.mean(cross_entropy_weighted, axis=-1)

Make sure you also split the stacked y_true_weights in your metrics function as well, ie:

def my_metric_fn(y_true_weights, y_pred):
    # split ground truth and weight map channels, each now of shape (H,W,1)
    y_true, weight_map = tf.split(y_true_weights, num_or_size_splits=2, axis=-1)
    ...calculate metrics using y_true as per usual, weight_map is not used.

For sanity, try to apply the splitting logic, but return regular mean cross entropy without applying wmap. When results match previous values you'll know all wires are in place.

Apologies in advance for this being borderline too programming-heavy and not quite machine-learning focused for this forum.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.