Research work that have solved problems using neural networks, simply state the structure of their networks in their research papers. No explanation is generally given about what led them to that structure (for example, AlphaGo's neural networks).
It can't be trial and error, since with huge networks the search space is huge; one can't simply try a lot of options and get to the correct structure.
And, it can't be fully deterministic either; some trial and error is involved, but not much. I've been taught in college that experts on neural networks make educated guesses of what to try.
I want to know what those educated guesses are and how is it driven. There must be some base principle/intuition or some data analysis on the given dataset, that can help us make educated guesses of our desired structure.
Please answer with small examples if possible. An example could be on a small enough dataset to make your point, or some problem that you solved using neural network (and knew how to design the network), or some research material where the actual process is explained. If these principles can be explained using XOR (classification) data, then please do so on that example..
One general idea is that instead of random weights, initialize the network weights using some domain knowledge about the problem. How is this done specifically?
Any more such ideas?