Bayesian regularized NNs over classical NNs I have seen a few research articles which claim that the classical neural networks usually lacks satisfactory generalization ability, which usually results in an imprecise predictions, and Bayesian regularized ANNs (BRANNs) are more robust than standard back-propagation nets and can reduce or eliminate the need for lengthy cross-validation. 
However these articles fall short of giving proper reasoning/justification for this claim.
In what ways, or for what particular purposes are BRANNs better than classical NNs? And why?
 A: The key problem with neural nets tends to be preventing over-fitting.  Bayesian regularisation (which restricts the magnitude of the weights) is one approach to this, structural stabilisation (i.e. restricting the number of hidden nodes and/or weights is another).  Neither approach is a panacea, and generally a combination of regularisation and structural stabilisation is better (which means you need cross-validation again to select the network architecture - using the Bayesian evidence for this is a bad idea as the evidence is biased as a result of its use in tuning the regularisation parameters and unreliable if there is any model miss-specification).  Which works best is essentially problem dependent, and the best way to find out is to try both and see (use e.g. cross-validation to estimate performance in an unbiased manner).
Also regularisation doesn't have to be Bayesian, you can choose how much to regularise the network using cross-validation instead.  One of the problems with Bayesian methods is that they can give bad results if the model is miss-specified, in which case cross-validation based regularisation methods may be more robust.
Another important point is that not all Bayesian neural network formulations are the same.  The Evidence framework of MacKay tends not to work to well for classification problems as the Laplace approximation that it uses doesn't work very well for skewed posterior distributions for the weights.  The MCMC approach of Radford Neal is likely to work better for these tasks, but is computationally expensive and assessing convergence etc. is not as straightforward.
However, neural network models are rather fiddly to get right and in practice it is easier to get good generalisation performance from kernel methods or Gaussian processes, so I would use them instead for most tasks, especially if there is relatively little training data.
I did a very extensive empirical study on this recently, but I need to find a journal that will accept empirical studies of interest to practitioners, but with very little new research content.
A: You use BRANNs for the same purposes as regular ANNs, typically classification and regression. As Dikran Marsupial says, the are better because they are more robust against overfitting, and allows you to work with higher number of neurons without running into overfitting. Besides, it provides you with error bars on the outputs, that is, you have a measure of the confidence of each of the outputs.
Nevertheless, new techniques like dropout and maxout seem to have overriden this technique, both because they are easier to use and yield better results. Here dropout is showed to perform scaling and regularization in certain sense.
Still, if you are interested on the details, you may check the papers by David MacKay (the guy who won some competitions with this technique).
