I have implemented Soft Plus for my training research with GoLang.

$$ \frac{1}{1+e^{-x}} $$


func SoftPlus(x float64) float64 {                                                     
    return (float64(1) / (float64(1) + math.Exp(-x)))

math.Exp(-x) returns 0 or infinity with large values of x (actually +/-1000 and greater/lesser, if negative)

The first solution which came to my mind is:

$$ \frac{1}{1+e^{-1*(1/x)}} $$

But it seems too naive and still has very low precision of probability.

Does exist any kind of Soft Plus optimization for such a case?

  • $\begingroup$ I am not familiar with the code you are using but for large positive values of x the function you are computing should be close to 1. $\endgroup$ – Michael R. Chernick Apr 19 '17 at 12:14
  • $\begingroup$ @MichaelChernick, yep I know but in a way it implemented and with large incoming values it returns 0 or 1, not close but exactly 0 or 1. That what I want to solve in a smarter way than I did it now. $\endgroup$ – I159 Apr 19 '17 at 12:27
  • $\begingroup$ You probably have a coding problem. I don't know how to debug your code. $\endgroup$ – Michael R. Chernick Apr 19 '17 at 12:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.