Both have the same equation : the logistic unit. Sigmoid output a ral-valued number between 0 and 1 and Stochastic binary neuron a probability between 0 and 1 too.
Apart from the name/type given to the output (probability or real-valued number) what is the difference between this two types of neurons ?
The only difference is that :
The sigmoid neuron is able to output values between 0 and 1.
The stochastic binary neurons only can output either a 0 or a 1 with the probability given by the sigmoid activation function.
Stochastic Binary is not a smooth function, it is like a step function where as Sigmoid is a smooth function. Although at extreme ends, large positive and large negative Sigmoid give a similar behavior as Stochastic but on moderate values it transits slowly. So a small change in the weights and bias will cause small change in the output, whereas this is not the case with Stochastic Binary neurons.
Refer to this book by Michael Neilson and I'm sure your confusion will be blown away. He explains it so well.
