custom loss function to optimize payoff via binary decision

Question

I have written a custom loss function that is supposed to optimize a payoff via a binary decision. However, the neural networks is struggling to convert, and I'm suspecting that there's something wrong with this function.

The payoff looks as follows. Imagine for each sample the neural network has to decide whether to engage or not (True/False). Each sample has some explanatory variables and an individual payoff (p) that is known. The reward or loss is then as follows:

• true positive: payoff (as passed in via variable over a second dimension in y_true). Payoff is a vector and is different for every sample.
• true negative: 0
• false positive: -1
• false negative: 0 (but has an opportunity cost of payoff)

The custom keras loss function looks as follows (I'm passing in payoffs via y_true, but split it out at the beginning. This is necessary as the keras payoff function only allows two parameters):

def custom_cross_entropy(y_true, y_pred):
    # y_true has the payoffs in the last dimension
    y_true, payoffs = splitter(y_true)

    tp_weight = K.abs(payoffs)  # true positive has a positive payoff as described in the payoff vector, it's different for every item
    tn_weight = 0  # true negative has a payoff of 0
    fp_weight = 1  # cost of 1 for false positive
    fn_weight = K.abs(payoffs)  # false negative has an opportunity cost of payoff (vector)

    loss = -K.mean(fn_weight * y_true * K.log(y_pred + 1e-7) +  # false negative
                   fp_weight * (1 - y_true) * K.log(1 - y_pred + 1e-7)  # false positive


def splitter(y_true):
    payoffs = y_true[:, 1]
    payoffs = K.expand_dims(payoffs, 1)
    y_true = y_true[:, 0]
    y_true = K.expand_dims(y_true, 1)
    return y_true, payoffs

What could be wrong with this payoff function? I only use the false negative and false positives. Is that correct? What else could be the problem? How would the loss function differ, if true negatives had a payoff of x? It seems that only using the false true and negatives is not enough. Any suggestions are appreciated.

Answer 1

There seems to be an error in the way you have designed the false negative side of this loss function,

Check out these graphs below on what happens for the respective targets, 0's and 1's when applied to the above function.

Here is the updated defination,

def custom_cross_entropy(y_true, y_pred):
    y_true, payoffs = splitter(y_true)
    
    # https://github.com/tensorflow/tensorflow/blob/v2.3.1/tensorflow/python/keras/backend.py#L4826
    y_pred = K.clip(y_pred, K.epsilon(), 1 - K.epsilon())

    term_0 = (1 - y_true) * K.log(1 - y_pred + K.epsilon()) # when you don't bet
    term_1 = y_true * K.log(y_pred + K.epsilon())  # when you bet

    return -K.mean(term_0 * K.abs(payoffs) + term_1 * K.abs(payoffs), axis=1)