For binary classification task, with samples labeled $y=0$ and $y=1$, a neural network has one output node with sigmoid activation function, producing predictions $\hat{y}\in(0;1)$. Is the following loss function usable or is it flawed (mathematically or numerically)? $$ L(y, \hat{y}) = \begin{cases} 1-\hat{y}, & \text{if $y=1$} \\ \hat{y}, & \text{if $y=0$} \end{cases}$$

The final objective is to predict $1$ (take an action) only when confident enough - "cherry-picking". Predicting $0$ incurs no cost but also has a zero utility. I think that translating it straight into a loss function woudn't help training a neural network, just like 0-1 loss.

I started with binary cross entropy and also considered using focal loss. However, the lowest achievable error rate is high (just below $\frac{1}{2}$), it is just not feasible to make a confident prediction. BCE seems to overly penalize being even appropriately somewhat confident. So I thought - why not trying a loss function which is even closer to the quantity I truly want to optimize.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.