What are the rules of designing a good convolutional neural (such as VGG , Inception  Resnet, DenseNet and so on...) network from scratch? What would be a recipe to start designing your own architecture?
Many meta-learning and architecture search algorithms (e.g, ) break-down the architecture design task into a simpler form. I wonder if a more general instruction can be gathered for researchers to follow.
I would appreciate it if you can provide a step-by-step guideline. Let's see if we can gather some useful tips and put them into test in here!
I will then follow these instructions and design a new architecture and will post the results in here.
 Very deep convolutional networks for large-scale image recognition K Simonyan, A Zisserman - arXiv preprint arXiv:1409.1556, 2014
 Szegedy, Christian, et al. "Inception-v4, inception-resnet and the impact of residual connections on learning." AAAI. Vol. 4. 2017.
 He, Kaiming, et al. "Deep residual learning for image recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
 Huang, Gao, et al. "Densely connected convolutional networks." Proceedings of the IEEE conference on computer vision and pattern recognition. Vol. 1. No. 2. 2017.
 Zoph, Barret, and Quoc V. Le. "Neural architecture search with reinforcement learning." arXiv preprint arXiv:1611.01578 (2016).