I'm teaching myself about neural networks to implement a simple one in C++ and have come to a roadblock. I am just starting to understand the math behind gradient descent and some of the other topics involved however what I don't understand is the exact implementation.
Assume I have a network with a depth of 3, input/hidden/output, after my hidden layer calculates the sigmoid from the weigted/biased value sent to each neuron does a hidden neuron pass on the sigmoid value to the next dendrite in the chain or does it convert it to a binary output? My assumption is that it converts it to binary but I keep hitting walls and questioning myself.
If both are possible which one is more efficient in terms of training speed? Forgive me if I have worded this poorly, like I stated I am still learning.