I am looking for a good book or an article concearning convolutional neural nets, especially their architecture. I like the http://deeplearningbook.org but it does not provide any information on the application, e.g. on how the size of filters should be chosen. Actually I am looking for a serious source for statements like that:

"It is common to increase the number of filters for deeper convolutional layers."

"A common choice for the max pooling layer is a filter of size $2\times 2$ and stride $2$."

There are tons of really good blogs on this topic (that's where I got my information from), but I need some citable sources for my thesis. It should not be too case-specific, especially as I don't use the CNNs on images what most articles seem to do.

EDIT: It would be great if the sources had an open access..

  • 1
    $\begingroup$ Other than Goodfellow's book, which you already mention, there isn't really a good textbook for CNNs. The effective techniques are too new, really just the last ten years, so reading the original papers is the best option. Here is an article which gives a good summary of the landmark papers - AlexNet, VGG-16, ResNet, etc., but even that is a few years old now. $\endgroup$ – olooney May 10 at 14:46
  • $\begingroup$ A good approach is to pick one of the open data sets - ImageNet, MNIST, Google's Street View House Numbers, COCO, etc., and read the papers for models which achieved state-of-the-art at one point or another. For example, ResNet is famous because it won ILSVRC and COCO in 2015 with a novel technique (residual blocks). YOLO is state-of-the-art on COCO... or at least it used to be. And so on. $\endgroup$ – olooney May 10 at 14:54
  • $\begingroup$ Oh, and statements like "a common choice for max pooling is a 2x2 kernel with stride 2" don't really have any support beyond statements like "VGG-16 and YOLO used 2x2s2 max-pooling every other layer, and those work great! Lots of other architectures use it to!" It's tribal knowledge, not quite cargo cult because there are empirical results, but there's no proof or anything. For example, ResNet does not use max-pooling, and doesn't seem to suffer from it. Since it's not a solved problem, but an open area of research, take literally all advice and "best practices" with a grain of salt. $\endgroup$ – olooney May 10 at 15:03
  • $\begingroup$ Thanks for your comment! Yeah it is a pity that the onyl justification one can give is "it seems to work well" :D A nice book I found is by the Keras founder, François Chollet, it is called Deep Learning with Python. It is focused on application and he gives nice general recommendations $\endgroup$ – msloryg May 13 at 8:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.