3
$\begingroup$

Is there any problem if we use too many hidden layers in Neural Network? Can anyone simply describe what problems can occur if we have too many hidden layers.

$\endgroup$

2 Answers 2

6
$\begingroup$

The gradient diminishes fast in very deep architectures, so the usual style of backpropagation doesn't work well. Backpropagated errors become very small after a few layers, which makes learning ineffective. This can be solved by pretraining, e.g. starting with initial weights that are close to the final solution.

$\endgroup$
1
$\begingroup$

If we increase the number of hidden layers then the neural network complexity increases. Moreover many application can be solved using one or two hidden layer. But for multiple hidden layers, proportionality plays a vital role. Also if hidden layer are increased then total time for training will also increase. Overfitting may arise due to network complexity.

But if you aim is to achieve accuracy then multiple hidden layers can do that.

$\endgroup$
1
  • $\begingroup$ W/ so many free parameters (due to the multiple hidden layers) you will need a lot of data or the "accuracy" will only be due to overfitting. You will need to cross validate to get an unbiased estimate of the accuracy you really have. $\endgroup$ Sep 9, 2014 at 13:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.