0
$\begingroup$

I'm building a binary classifyer that has an unequal payoff given the following cass:

  • $Y_{pred}=Y_{actual}= True$: payoff is +x*100
  • $Y_{pred}$$\ne$$Y_{actual}= True$: payoff is -x*100
  • $Y_{pred}$$\ne$$Y_{actual}= False$: payoff is -1
  • $Y_{pred}=Y_{actual}= False$: payoff is +1

In other words, there are two possible courses of action: True for action 1 and False for action 2.

I'm looking at two approaches to implement it:

  1. I could mse as loss function with two output neurons, assign the payoffs directly to $Y_{actual}$, with the neural network predicting the payoffs for each of the actions.

    • I assume I would then just look for which of the two neurons predicts a higher payoff and use liner function in the last activation layer. Correct?
  2. I could create a custom loss function that will calculate the respective payoffs as described above and then use binary cross entropy on that?

Which of those two approaches would be preferred? Should they lead to the same result, giving a recommendation for each sample which action is preferable?

$\endgroup$
1
$\begingroup$

You cannot use your "payoff" function as a loss for your network because it is not differentiable. Instead, you can use a differentiable function which has outputs close to your evaluation metric (payoff).

As you are predicting a binary variable, the way to go is the binomial cross-entropy. The loss function would look like:

$$\mathcal{L}(\mathbf y, \mathbf c, \mathbf t)=-\frac {1}{N}\sum_n 100c_n t_n\log y_n + (1-t_n)\log (1-y_n),$$

where $\mathbf y$ are the predictions, $\mathbf t$ are the targets and $\mathbf c$ are the individual costs for each sample. You can see it is giving the false negative predictions $100c_n$-times higher penalty than false positives.

It is possible that you will have to implement this loss yourself, but it should be quite easy.

Using MSE for classification does not make much sense.

$\endgroup$
  • $\begingroup$ In my case each sample would need to have an individual weight. Also, cross entropy with weights only seems to offer a coefficient for positive cases - unclear to me what that means as I have basically 4 different payoffs that need to be weighted. $\endgroup$ – Nickpick Feb 7 '18 at 23:28
  • $\begingroup$ What is "x" in your "payoff"? The weight coefficient for each sample? $\endgroup$ – Jan Kukacka Feb 8 '18 at 10:40
  • $\begingroup$ Yes that's correct. So I would need the samples weighted but also dependent on the outcome (if correctly matched or not). $\endgroup$ – Nickpick Feb 8 '18 at 10:43
  • $\begingroup$ Check the updated answer. Hope it solves your problem. $\endgroup$ – Jan Kukacka Feb 8 '18 at 11:01
  • 1
    $\begingroup$ I guess not, but you can use the tensorflow sigmoid_cross_entropy_with_logits source code and modify it for your purpose. $\endgroup$ – Jan Kukacka Feb 8 '18 at 12:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.