This question already has an answer here:

What is the use of Hidden layers in Neural Networks? How do we know the role of each hidden layer?

marked as duplicate by gung Dec 20 '17 at 18:50

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

Neural networks are universal function approximators. To approximate any function, you need to be able to model it in a non-linear way.

If you consider Neural Network with just input-output layer, then it is just linear approximation (i.e. visually equal to drawing a straight line to divide the example-categories). Hidden layers allow introducing non-linearities to function.

E.g. think about Taylor series. You need to keep adding polynomials to approximate the function. You can draw an analogy (although weak) between adding the polynomials and adding the hidden layers in the neural network.

The role of each hidden layer cannot be easily known beforehand.

Having too many hidden layers will make the Neural network to overfit the function ("high variance"). Having not enough hidden layers will make the Neural network to underfit the function ("high bias").

  • I am confused of one thing. What if I have just input-output layer without any hidden layers and I use sigmoid activation function to determine output units. In that case it will we non-linear. Then why hidden layer in this case? – user195278 Dec 11 '17 at 7:19
  • 1
    You are describing a logistic regression. The decision boundary is still a straight line. If you added higher order terms to parameters, then you could draw non-linear decision boundary.Maybe this helps: holehouse.org/mlclass/06_Logistic_Regression.html (it seems to be notes from Andrew Ng's Machine learning -course). – Tuomas Dec 11 '17 at 7:58
  • Yup. Thanks. Now it makes sense....Got it. One last doubt. How do we determine the values of weights for mapping? – user195278 Dec 11 '17 at 8:16
  • For that I suggest to investigate the backpropagation -algorithm. – Tuomas Dec 11 '17 at 8:33

Not the answer you're looking for? Browse other questions tagged or ask your own question.