What is the use of Hidden layers in Neural Networks? How do we know the role of each hidden layer?
$\begingroup$
$\endgroup$
0
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
Neural networks are universal function approximators. To approximate any function, you need to be able to model it in a non-linear way.
If you consider Neural Network with just input-output layer, then it is just linear approximation (i.e. visually equal to drawing a straight line to divide the example-categories). Hidden layers allow introducing non-linearities to function.
E.g. think about Taylor series. You need to keep adding polynomials to approximate the function. You can draw an analogy (although weak) between adding the polynomials and adding the hidden layers in the neural network.
The role of each hidden layer cannot be easily known beforehand.
Having too many hidden layers will make the Neural network to overfit the function ("high variance"). Having not enough hidden layers will make the Neural network to underfit the function ("high bias").
-
$\begingroup$ I am confused of one thing. What if I have just input-output layer without any hidden layers and I use sigmoid activation function to determine output units. In that case it will we non-linear. Then why hidden layer in this case? $\endgroup$ Dec 11, 2017 at 7:19
-
1$\begingroup$ You are describing a logistic regression. The decision boundary is still a straight line. If you added higher order terms to parameters, then you could draw non-linear decision boundary.Maybe this helps: holehouse.org/mlclass/06_Logistic_Regression.html (it seems to be notes from Andrew Ng's Machine learning -course). $\endgroup$– TuomasDec 11, 2017 at 7:58
-
$\begingroup$ Yup. Thanks. Now it makes sense....Got it. One last doubt. How do we determine the values of weights for mapping? $\endgroup$ Dec 11, 2017 at 8:16
-
$\begingroup$ For that I suggest to investigate the backpropagation -algorithm. $\endgroup$– TuomasDec 11, 2017 at 8:33