I have checked a lot of questions here and in other websites. What I concluded is that there is no rules for choosing the right number of hyper-parameters in CNN, all what can we do is just trying many cases, and select the best one. Thus, I still wondering if we can choose a preliminary number of those hyper-parameters (point to start) based on the type of input images (histological medical images in my case), the size and the number of input data, the architecture of the network, etc.

Glad to get a more additional explanation / helpful links. Thanks


Some good rules of thumb for selecting hyperparameters are:

Number of layers: between 5 and 100, depending on how difficult the task is. You can start with 5 and work your way up. If you want more than 10 layers, probably use skip/residual connections of some form.

Number of kernels: start with a small number (16, 32, or 64) on your first layer, and double the number every time your feature map shrinks by 2 on each side (due to either a pooling layer or a strided conv).

Kernel size: 3x3 everywhere except the first layer, where you can use something slightly larger.

  • $\begingroup$ Can I use a portion of input data to find the best values of these hyper-parameters in order to speed up the process, or I have to test them with all the available data? $\endgroup$ – Khalil Meg Apr 20 '19 at 17:28
  • 1
    $\begingroup$ not really, i think at that point you're just going to have to start doing hyperparameter optimization the hard way $\endgroup$ – shimao Apr 21 '19 at 19:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.