1
$\begingroup$

What is the best way to maximize a payoff based on a binary decision with using a neural network?

I understand that with a binary cross-entropy loss function I can bring a network with a sigmoid output function to make the right binary decision. But when each of the decision in the sample may have a different payoff, how would i best weight those payoffs or losses during the trainig?

Is there any recommended way to do that? Any suggestions are appreciated.

$\endgroup$
3
$\begingroup$

You can just try to maximize the expected value, per sample.

Given $x$, for each decision $k\in\{1,2,\cdots,K\}$, write the payoff as $P(x,k)$. Then for a single sample, the payoff would be $\sum_{k=1}^KP(x,k)p_k(x)$, where $p_k(x)$ is the predicted class probability (in your case, a softmax).

You could also minimize regret: the difference between the top choice and your models choice. It's difficult to train based on the "highest probability" because of discontinuities, so you can substitute the expected value as a proxy. This can be written as:

$\sum_{k=1}^K |P(x,k_{max}(x))-\sum_{i=1}^kP(x,k)p_k(x)|^2$

$\endgroup$
3
  • $\begingroup$ Interesting. But if $p_k(x)$ predicts the class probability and I'm using let's say a sigmoid output and round it to 0 or 1 for each class, would that still work as a loss function. Is it still differentiable dispite the jumps, or am I not allowed to round? $\endgroup$
    – Nickpick
    Dec 11 '17 at 19:13
  • $\begingroup$ @Nickpick: In terms of rounding, this is a problem that occurs even without the payoff. This is where ROC analysis comes in. In this case, you can set rounding thresholds (if above x, then =1, below x then = 0), which will show you the expected payoff for each threshold. $\endgroup$
    – Alex R.
    Dec 11 '17 at 20:14
  • $\begingroup$ So to be clear, in the above model, the $p_k(x)$ are unrounded. To make a choice of which you want, you can, for example, pick the highest one. $\endgroup$
    – Alex R.
    Dec 11 '17 at 20:19
0
$\begingroup$

@Alex R equation in Keras,

def splitter(y_true):
    payoffs = y_true[:, 1]
    payoffs = K.expand_dims(payoffs, 1)
    y_true = y_true[:, 0]
    y_true = K.expand_dims(y_true, 1)
    return y_true, payoffs

def custom_odds_loss(y_true, y_pred):
    y_true, payoffs = splitter(y_true)
    
    # https://github.com/tensorflow/tensorflow/blob/v2.3.1/tensorflow/python/keras/backend.py#L4826
    y_pred = K.clip(y_pred, K.epsilon(), 1 - K.epsilon())
    
    term_0 = K.sum((1 - y_true) * K.abs(payoffs) * (1 - y_pred), axis=1)  # Cancels out when target is 1 
    term_1 = K.sum(y_true * K.abs(payoffs) * y_pred, axis=1) # Cancels out when target is 0
    return K.square(K.abs(K.max(payoffs) - term_1 - term_0))

enter image description here enter image description here

My variation to the above equation,

Useful if every batch resembles an independent event of observations and that the designated payoff is one observation per batch

def custom_odds_loss(y_true, y_pred):
    """
    K.max(payoffs * y_true) - ... ensures higher penalty where the **winner** observation has higher payoff
    """
    y_true, payoffs = splitter(y_true)
    
    # https://github.com/tensorflow/tensorflow/blob/v2.3.1/tensorflow/python/keras/backend.py#L4826
    y_pred = K.clip(y_pred, K.epsilon(), 1 - K.epsilon())
    term_0 = K.sum((1 - y_true) * K.abs(payoffs) * (1 - y_pred), axis=1)  # Cancels out when target is 1 
    term_1 = K.sum(y_true * K.abs(payoffs) * y_pred, axis=1) # Cancels out when target is 0
    return K.square(K.abs(K.max(payoffs * y_true)  - term_1 - term_0))
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.