# Soft Plus activation function with large values

I have implemented Soft Plus for my training research with GoLang.

$$\frac{1}{1+e^{-x}}$$

Implementation:

func SoftPlus(x float64) float64 {
return (float64(1) / (float64(1) + math.Exp(-x)))
}


math.Exp(-x) returns 0 or infinity with large values of x (actually +/-1000 and greater/lesser, if negative)

The first solution which came to my mind is:

$$\frac{1}{1+e^{-1*(1/x)}}$$

But it seems too naive and still has very low precision of probability.

Does exist any kind of Soft Plus optimization for such a case?

• I am not familiar with the code you are using but for large positive values of x the function you are computing should be close to 1. – Michael R. Chernick Apr 19 '17 at 12:14
• @MichaelChernick, yep I know but in a way it implemented and with large incoming values it returns 0 or 1, not close but exactly 0 or 1. That what I want to solve in a smarter way than I did it now. – I159 Apr 19 '17 at 12:27
• You probably have a coding problem. I don't know how to debug your code. – Michael R. Chernick Apr 19 '17 at 12:29