This is an interesting question - in a lot of ways, it's really the question in Machine Learning.
The short version of the answer is something like this: your choice of model is very important, but there are no silver bullets or magic hammers to be had; with a few notable exceptions, it's worth thinking of all of the techniques that you've learned as tools to be used in the right situation. The actual choice that takes place usually involves some analysis, some visualization, some intuition, some experience, and a little bit of voodoo (which is to say, luck).
I'll answer you in two parts: One, I'll give examples of problems suited to particular models, and two, I'll outline some approaches that I take when I'm first setting things up.
Consider the problem of image recognition in a complex environment. It's not an easy problem to solve; it can often deal with rich features and different levels of abstraction. In this case, one might try a neural network of some variety(let's say something fancy, like a convolutional net): this has the advantage of being a very expressive model, which might be just the power tools needed to model the complex interrelationships of the variables. Of course, the penalty paid is that the algorithm can be rather expensive; however, it might be that this slowness is justified given the advantage conferred by the richness of the model.
Now, consider the very different problem of finding the general trend in noisy time-series data over a long span of time; let's say our choices are "going up" or "going down". In this case, one might be tempted to use a very complicated solution like the neural net above; as we mentioned, it would be a powerful tool, which could explain complicated relationships in the data. On the other hand, for a question with such a limited scope, it might be useful to simply use OLS regression and look at the slope of a linear equation fit to the data. It would be quick and dirty, but it would probably do the job. In this situation, the Neural Network might be monstrous overkill.
You'll probably notice that there is a lot of "maybe" and "may" and "it's possible" in my answer above. Unfortunately, there tend to be no hard and fast rules. When I think about what model to use, I think of the following criteria, which at least puts me in the right direction (hopefully):
What are the computational constraints? (For example, if I need to provide fast estimates to a user on a webpage based on dynamic inputs, I won't be using a deep network).
What does the data look like when it's visualized? For regression problems, does the input resemble any mathematical functions I know? Can any transformations be applied to it? For clustering, are there any obvious patterns? How are the clusters shaped, approximately?
How sparse or dense is the data set? What is the propensity for overfitting? How easy would it be to perform cross-validation?
After that, it's a matter of using experience and doing some experimental tests, then improving on it. It's just as much an art as it is a science, which makes it as tough as it is exciting.
I hoped that helped; tell me if you have any questions. Good luck!
EDIT: As a short addendum, and in answer to the specific questions you posed: expressive models such as NN are useful when the data contains many rich relationships; however, the models you call "rigid" can be very useful when complexity is a major issue and/or when the general form of the data can be predicted in some way (for example, "the data looks like a linear function with some noise").
Edit edit: Wait, one more thing - I've never found the approach of "unleash all the algorithms and see what sticks" to be useful. In my opinion, the single most important skills one can cultivate in this field is to understand why certain problems are best solved by certain solutions; it'll save you huge amounts of time.