I am trying to implement a multi-layer deep neural network (over 100 layers) for image recognition. As far as i can understand each layer learns specific features.

I am feeding 100x100 pixel color RGB facial images to a DNN and trying to capture nose, eyes etc.

But how do i decide on number of layers on deep neural network?

I am using FANN c library for neural network.


As Yoshua Bengio, Head of Montreal Institute for Learning Algorithms remarks:

"Very simple. Just keep adding layers until the test error does not improve anymore."

A method recommended by Geoff Hinton is to add layers until you start to overfit your training set. Then you add dropout or another regularization method.

  • $\begingroup$ how to decide on number of neurons per layer? $\endgroup$ – pbu Jan 24 '16 at 20:31
  • $\begingroup$ For your task, your input layer should contain 100x100=10,000 neurons for each pixel, the output layer should contain the number of facial coordinates you wish to learn (e.g. "left_eye_center", ...), and the hidden layers should gradually decrease (perhaps try 6000 in first hidden layer and 3000 in the second; again it's a hyper-parameter to be optimised and is very dependent on dataset size). This tutorial regarding your specific task should be useful (in particular, a CNN would likely be much better than just a standard fully-connected DNN). $\endgroup$ – andyandy Jan 25 '16 at 13:00

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.