As an introductory text to all the issues you name, I would recommend the deep learning book. It provides a broad overview of the field. It explains the role each of those parameters play.
In my opinion is very helpful to read about some of the most popular architectures (resnet, inception, alex-net), and extract the key ideas leading to the design decisions. After reading the aforementioned book.
In the syllabus of the lectures you refer to, it is explained in great detail how the convolution layer adds a big number of parameters (weights, biases) and neurons. This layer, once trained, it is able to extract meaning patterns from the image. For lower layers those filters look like edge extractors. For higher layers, those primitive shapes are combined to describe more complex forms. Those filters involve a high number of parameters, and a big issue of the design of deep networks in how to be able to describe complex forms and still be able to reduce the number of parameters.
Since neighboring pixels are strongly correlated (specially in lowest layers), it makes sense to reduce the size of the output by subsampling (pooling) the filter response. The further apart two pixels are from each other, the less correlated. Therefore, a big stride in the pooling layer leads to high information loss. Loosely speaking. A stride of 2 and a kernel size 2x2 for the pooling layer is a common choice.
A more sophisticated approach is the Inception network (Going deeper with convolutions) where the idea is to increase sparsity but still be able to achieve a higher accuracy, by trading the number of parameters in a convolutional layer vs an inception module for deeper networks.
A nice paper that provides hints on current architectures and the role of some of the design dimensions in a structured, systematic way is SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size. It builds on ideas introduced in the models previously mentioned.
