1
$\begingroup$

I am trying to implement a multi-layer deep neural network (over 100 layers) for image recognition. As far as i can understand each layer learns specific features.

I am feeding 100x100 pixel color RGB facial images to a DNN and trying to capture nose, eyes etc.

But how do i decide on number of layers on deep neural network?

I am using FANN c library for neural network.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
10
$\begingroup$

As Yoshua Bengio, Head of Montreal Institute for Learning Algorithms remarks:

"Very simple. Just keep adding layers until the test error does not improve anymore."

A method recommended by Geoff Hinton is to add layers until you start to overfit your training set. Then you add dropout or another regularization method.

Improve this answer
$\endgroup$
2
  • $\begingroup$ how to decide on number of neurons per layer? $\endgroup$
    – pbu
    Jan 24, 2016 at 20:31
  • $\begingroup$ For your task, your input layer should contain 100x100=10,000 neurons for each pixel, the output layer should contain the number of facial coordinates you wish to learn (e.g. "left_eye_center", ...), and the hidden layers should gradually decrease (perhaps try 6000 in first hidden layer and 3000 in the second; again it's a hyper-parameter to be optimised and is very dependent on dataset size). This tutorial regarding your specific task should be useful (in particular, a CNN would likely be much better than just a standard fully-connected DNN). $\endgroup$
    – andyandy
    Jan 25, 2016 at 13:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.