I am trying to create a Deep Belief Network (DBF) for a binary classification problem. The nolearn package provides a good library for implementing them. I see that there are very many parameters to learn and tune a DBF.

The possible list of parameters are:

  1. The number of hidden layers and the number of neurons in each.
  2. Learning rate
  3. learn_rate_decays - The number with which the learn_rate is multiplied after each epoch of fine-tuning.
  4. Momentum
  5. Number of epochs to train (with backprop).
  6. learn_rates_pretrain – A list of learning rates for pretraining.
  7. momentum_pretrain – Momentum for pre-training.
  8. L2 costs per weight layer.
  9. Dropouts per weight layer.
  10. Momentum for pre-training.
  11. L2 costs per weight layer, for pre-training.
  12. Number of epochs to pre-train (with CDN).
  13. Scale of the randomly initialized weights.
  14. Number of nonzero incoming connections to a hidden unit.
  15. Output activation function. (can be any of Sigmoid, Linear, Softmax)

Among these what are the most important parameters to tune? Do I have to tune all the parameters?

  • $\begingroup$ This is potentially a reasonable & non-software-specific question. However, I doubt anyone who doesn't use this software (what?) will read or be familiar w/ this. Can you make this (1) software-neutral, (2) self-contained, & (3) more intelligible? $\endgroup$ – gung Feb 20 '17 at 16:21
  • $\begingroup$ @gung Thanks for looking into the question. Have modified the question. $\endgroup$ – prashanth Feb 21 '17 at 6:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.