1
$\begingroup$

I have no problem to train my neural network with categorical_crossentropy as loss but when I do the same with f1, it just doesn't progress :

Epoch 1/9 1029/1029 [==============================] - 4384s 4s/step - loss: 0.9470 - f1: 0.9470

Epoch 3/9 1029/1029 [==============================] - 1401s 1s/step - loss: 0.9413 - f1: 0.9413 - val_loss: 0.9938 - val_f1: 0.9938

Epoch 7/9 1029/1029 [==============================] - 1405s 1s/step - loss: 0.9270 - f1: 0.9270 - val_loss: 0.9932 - val_f1: 0.9932

Here's my implementation of f1: def f1(y_true, y_pred): def recall(y_true, y_pred): """Recall metric.

    Only computes a batch-wise average of recall.

    Computes the recall, a metric for multi-label classification of
    how many relevant items are selected.
    """
    true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
    possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)))
    recall = true_positives / (possible_positives + K.epsilon())
    return recall

def precision(y_true, y_pred):
    """Precision metric.

    Only computes a batch-wise average of precision.

    Computes the precision, a metric for multi-label classification of
    how many selected items are relevant.
    """
    true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
    predicted_positives = K.sum(K.round(K.clip(y_pred, 0, 1)))
    precision = true_positives / (predicted_positives + K.epsilon())
    return precision
precision = precision(y_true, y_pred)
recall = recall(y_true, y_pred)
return 2*((precision*recall)/(precision+recall+K.epsilon()))

Do you have an idea of what might be the problem ?

$\endgroup$
  • $\begingroup$ do you always get constant loss (without slight change)? $\endgroup$ – itdxer May 22 at 16:06
  • $\begingroup$ Constant I don't know but it's always very low. Why ? $\endgroup$ – Ouallez May 23 at 18:18
  • $\begingroup$ if it constant than it means that, probably, weights don't get updated and it's possible when gradients are always 0. I believe that f1-score should produce 0 gradient $\endgroup$ – itdxer May 24 at 7:14
0
$\begingroup$

F1 score is non-differentiable, thus cannot be used as a loss function.

$\endgroup$
  • 1
    $\begingroup$ This is being automatically flagged as low quality, probably because it is so short. At present it is more of a comment than an answer by our standards. Can you expand on it? You can also turn it into a comment. $\endgroup$ – gung - Reinstate Monica May 22 at 16:33
  • $\begingroup$ Is it possible to round the curve artificially around the sharp turns to make the function differentiable ? Can we replace the 'round' in the f1 function by something else ? $\endgroup$ – Ouallez May 24 at 12:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.