How can we determine the appropriate number of hidden layers, kernels in convolutional neural network (CNN)?

Question

I have checked a lot of questions here and in other websites. What I concluded is that there is no rules for choosing the right number of hyper-parameters in CNN, all what can we do is just trying many cases, and select the best one. Thus, I still wondering if we can choose a preliminary number of those hyper-parameters (point to start) based on the type of input images (histological medical images in my case), the size and the number of input data, the architecture of the network, etc.

Glad to get a more additional explanation / helpful links. Thanks

Answer 1

Some good rules of thumb for selecting hyperparameters are:

Number of layers: between 5 and 100, depending on how difficult the task is. You can start with 5 and work your way up. If you want more than 10 layers, probably use skip/residual connections of some form.

Number of kernels: start with a small number (16, 32, or 64) on your first layer, and double the number every time your feature map shrinks by 2 on each side (due to either a pooling layer or a strided conv).

Kernel size: 3x3 everywhere except the first layer, where you can use something slightly larger.