I would like to discuss chapter that comes from Foreign-Exchange-Rate Forecasting With Artificial Neural Networks.

This chapter (see screenshot) describes a binary classifier made from neural network output values (this case predicts binary output). I understand meaning of these formulas but what I do not understand is equation 12.4 in the sense how can be normalized value $g_i^+(x)$ used as a probability value for positive outcome and $1 - g_i^-(x)$ for negative outcome prediction?

Q: How is strong connection between predicted value from neural network and probability of predicted binary outcome?

Edit1: @mhdadk: I am aware of this work with probability in binary output, my question was more about how relevant is the normalization with the sigmoid logical function itself to produce confidence / probability measure (probability density).

I'm thinking that, for example, if the input to normalization function is in the interval approx. $(-\infty, -5)$ or $(5,+\infty)$ will basically still be normalized to the value of 1 (simply and figuratively speaking). With respect to sigmoid function everything except values from the interval $(-5,5)$ will be "truncated" to the value very near of 1. And outside the interval $(-5,5)$ there is infinitely more values than inside this interval.

How does the probability, after conversion to the normalized value $(0,1)$, is significant in this manner?

enter image description here

  • $\begingroup$ I think you are asking about the "squishing" behavior of the sigmoid function for extreme values, those beyond, say, -5 and 5. If so, what exactly do you want to know about these extreme values? By the way, this paper may offer the insight you are looking for. $\endgroup$
    – mhdadk
    Nov 7, 2021 at 21:48
  • $\begingroup$ You appear to be trying to do something that would be quite simple with a statistical regression model. And "binary classifier outcome" doesn't mean very much. Remember that classifiers are forced choice dichotomizers which is seldom what is really needed to solve a problem. See for example this. $\endgroup$ Nov 7, 2021 at 22:54

1 Answer 1


Given two classes $c_1$ and $c_2$, if $$ p(c_1 \mid x) = g_i^+(x) $$ and since $$ p(c_1 \mid x) + p(c_2 \mid x) = 1 $$ then \begin{align} p(c_1 \mid x) + p(c_2 \mid x) &= 1 \\ p(c_2 \mid x) &= 1 - p(c_1 \mid x) \\ p(c_2 \mid x) &= 1 - g_i^+(x) \end{align}

  • $\begingroup$ Thanks for the answer, I added clarification to my question. $\endgroup$
    – jnemecz
    Nov 7, 2021 at 21:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.