I am trying to implement a multi-layer deep neural network (over 100 layers) for image recognition. As far as i can understand each layer learns specific features.

I am feeding 100x100 pixel color RGB facial images to a DNN and trying to capture nose, eyes etc.

But how do i decide on number of layers on deep neural network?

I am using FANN c library for neural network.


1 Answer 1


As Yoshua Bengio, Head of Montreal Institute for Learning Algorithms remarks:

"Very simple. Just keep adding layers until the test error does not improve anymore."

A method recommended by Geoff Hinton is to add layers until you start to overfit your training set. Then you add dropout or another regularization method.

  • $\begingroup$ how to decide on number of neurons per layer? $\endgroup$
    – pbu
    Commented Jan 24, 2016 at 20:31
  • $\begingroup$ For your task, your input layer should contain 100x100=10,000 neurons for each pixel, the output layer should contain the number of facial coordinates you wish to learn (e.g. "left_eye_center", ...), and the hidden layers should gradually decrease (perhaps try 6000 in first hidden layer and 3000 in the second; again it's a hyper-parameter to be optimised and is very dependent on dataset size). This tutorial regarding your specific task should be useful (in particular, a CNN would likely be much better than just a standard fully-connected DNN). $\endgroup$
    – andyandy
    Commented Jan 25, 2016 at 13:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.