I have some good news and some bad news for you. The good news is that there are many problems for which we know which architecture works best, because of previous research. The bad news is that since today we don't have a good theory of generalization for Deep Networks, we lack theoretical guidance about how to select an architecture for a new problem (however, read here for some insights). Thus, in general the most honest answer is that "it's just a matter of understanding and experience". On the other hand, for some specific fields we can give more canned suggestions:
We know that the Convnet family of architectures works very well for image classification: LeNet, Alexnet, VGGNet, ResNets, etc. You can train a beefed-up version of LeNet on a non-GPU laptop, and that's a great way to start learning about them. I suggest you start from this Keras implementation
and try to improve it a bit, by reproducing in Keras the section Implementing a CNN in the TensorFlow layers API of this Jupyter notebook. By the way, I can't recommend the second edition of Sebastian Raschka's book highly enough - it's a great way to gain practical knowledge about Machine Learning and Deep Learning. Instead of wasting time reading multiple tutorials on the Internet, get the book - you'll get a more solid understanding of the subject, also because quite a few of the most cited blog posts on Convolutional Neural Networks are basically summaries of the first edition of the book.
If you want to train architectures which will perform well on realistic, big data sets (such as CIFAR-100 or ImageNet), you need to have access to a GPU cluster.
Natural Language Processing
Here we know that RNNs work well. Actually, the "simple" RNN architecture known as LSTM delivers much better results than most people would commonly expect, as shown in this paper: On the State of the Art of Evaluation in Neural Language Models. The paper also highlights a big limit of modern Deep Learning research: a lot of papers don't care enough about repeatability and reproducibility of results, and some results than are presented as the new state of the art, are instead just due to uncontrolled experimental variation. Again, Raschka's book can be quite useful to start learning about RNNs, together with the corresponding Jupyter notebook.
The general case
If you want to tackle a known problem, but maybe with a new data set (for example, you want to perform image classification of car parts because you work for a car manufacturer), you need to use model selection techniques, such as for example cross-validation. You build different networks (different number of layers, different activation functions, etc.) and choose the one with the smallest cross-validation error. Then, you retrain it on the full data set, and you use it for prediction.
However, since the number of alternatives can be prohibitive, you can use some automated machine learning frameworks which help you explore the space of possible networks, such as for example:
If you need to work on big data sets, these tools won't work (they are based on scikit-learn, so there's no support for GPUs, currently). You may have a look at this paper, Large-Scale Evolution of Image Classifiers: like the other one I linked, this one takes proper care to ensure repeatability of results.
If you want to attack a new problem (say, Neural Program Synthesis) for which we still have no idea of which architectures work best, probably your best bet is to attend NIPS and ICML (or stalk the right sections of arXiv), in the hope that someone has already tackled your problem.