# Neural Networks - Do all neurons in hidden layer activate?

Noob question.

Okay I am beginning with MLPs and machine learning.

Suppose that I have 2 hidden layers in an ANN that uses the sigmoid function.

So does that mean that after calculating the weighted sum of inputs from the input layer all neurons in the hidden layer will only output a value between 0 and 1 in each of the neurons ?

Or will they output a weighted sum in the hidden layers and ONLY when it goes to the output neurons will the activation function be used.

It's very unclear what you want, but I'll try.

Yes, the output of each sigmoidal neuron is strictly between 0 and 1 due to the definition of sigmoidal function. These outputs become input vector in the second layer, so yeah, in a sense they 'output a weighted sum in the hidden' layer. Is this what ou wanted?

• Yes. This what I wanted. Thanks. So let me just get it straight. After calculating the weighted sum a neuron passes that sum into a sigmoid(or any activation function) and calculates a value which becomes a input for the next layer. That means all neurons pass the weighted sum through an activation function, I mean there is no neuron that passes just the weighted sum to the next layer without the activation function, RIGHT ??? Jan 22 '16 at 13:04
• You can use the linear function, i.e. $f(x) = x$, it is even continuous, but what's the point?
– Alex
Jan 22 '16 at 13:05
• Exactly, so the activation is done at every hidden layer. You see I am not confused about the parameters. I am confused about the fact that after calculating the parameters is the final result passed through an activation function in the hidden layer(NOT the OUTPUT LAYER) Jan 22 '16 at 13:08
• 'after calculating the parameters'-?? I'm sorry, it's very unclear what you want. 'calculating the parameters', i.e. weights and hidden unit expressions are very different things
– Alex
Jan 22 '16 at 13:10
• Sorry after multiplying the inputs with the weights. Jan 22 '16 at 13:10