I understand that a radial basis function neural network (RBF) usually has 1 hidden layer, and it differs from a multi-layer perceptron (MLP) via its activation and combination functions among other things, but how do I decide when a data set/problem is better suited to an RBF instead of an MLP? Do I have to try both and compare the results every time?
You may use RBF networks in case you do not necessarily need to have multiple hidden layers in your model and more importantly, you want your model to be robust to adversarial noise/examples. The advantage of RBF networks is they bring much more robustness to your prediction, but as mentioned earlier they are more limited compared to commonly-used types of neural networks. However, commonly-used types of neural network models are highly vulnerable to adversarial noise and can make very wrong predictions when fed with such examples as their inputs. This is not the case in RBF networks which seems to be due to their non-linear nature of these networks. So it is a trade-off between higher accuracy in commonly-used types of neural networks or higher robustness in radial-basis function networks.