The best way to maximize a payoff based on a binary decision with neural network

Question

What is the best way to maximize a payoff based on a binary decision with using a neural network?

I understand that with a binary cross-entropy loss function I can bring a network with a sigmoid output function to make the right binary decision. But when each of the decision in the sample may have a different payoff, how would i best weight those payoffs or losses during the trainig?

Is there any recommended way to do that? Any suggestions are appreciated.

Answer 1

You can just try to maximize the expected value, per sample.

Given $x$, for each decision $k\in\{1,2,\cdots,K\}$, write the payoff as $P(x,k)$. Then for a single sample, the payoff would be $\sum_{k=1}^KP(x,k)p_k(x)$, where $p_k(x)$ is the predicted class probability (in your case, a softmax).

You could also minimize regret: the difference between the top choice and your models choice. It's difficult to train based on the "highest probability" because of discontinuities, so you can substitute the expected value as a proxy. This can be written as:

$\sum_{k=1}^K |P(x,k_{max}(x))-\sum_{i=1}^kP(x,k)p_k(x)|^2$

Answer 2

@Alex R equation in Keras,

def splitter(y_true):
    payoffs = y_true[:, 1]
    payoffs = K.expand_dims(payoffs, 1)
    y_true = y_true[:, 0]
    y_true = K.expand_dims(y_true, 1)
    return y_true, payoffs

def custom_odds_loss(y_true, y_pred):
    y_true, payoffs = splitter(y_true)
    
    # https://github.com/tensorflow/tensorflow/blob/v2.3.1/tensorflow/python/keras/backend.py#L4826
    y_pred = K.clip(y_pred, K.epsilon(), 1 - K.epsilon())
    
    term_0 = K.sum((1 - y_true) * K.abs(payoffs) * (1 - y_pred), axis=1)  # Cancels out when target is 1 
    term_1 = K.sum(y_true * K.abs(payoffs) * y_pred, axis=1) # Cancels out when target is 0
    return K.square(K.abs(K.max(payoffs) - term_1 - term_0))

My variation to the above equation,

Useful if every batch resembles an independent event of observations and that the designated payoff is one observation per batch

def custom_odds_loss(y_true, y_pred):
    """
    K.max(payoffs * y_true) - ... ensures higher penalty where the **winner** observation has higher payoff
    """
    y_true, payoffs = splitter(y_true)
    
    # https://github.com/tensorflow/tensorflow/blob/v2.3.1/tensorflow/python/keras/backend.py#L4826
    y_pred = K.clip(y_pred, K.epsilon(), 1 - K.epsilon())
    term_0 = K.sum((1 - y_true) * K.abs(payoffs) * (1 - y_pred), axis=1)  # Cancels out when target is 1 
    term_1 = K.sum(y_true * K.abs(payoffs) * y_pred, axis=1) # Cancels out when target is 0
    return K.square(K.abs(K.max(payoffs * y_true)  - term_1 - term_0))