What are the advantages of using a Bayesian neural network Recently I read some papers about the Bayesian neural network (BNN) [Neal, 1992], [Neal, 2012], which gives a probability relation between the input and output in a neural network. Training such a neural network is through MCMC which is different from the traditional back-propagation algorithm. 
My question are: What is the advantage of using such a neural network? More specifically, could you provide some examples which better fit BNN rather than NN? 
 A: Bayesian neural nets are useful for solving problems in domains where data is scarce, as a way to prevent overfitting. They often beat all other methods in such situations. Example applications are molecular biology (for example this paper) and medical diagnosis (areas where data often come from costly and difficult expiremental work). Actually, Bayesian nets are univerally useful and can obtain better results for a vast number of tasks but they are extremely difficult to scale to large problems. 
A: One advantage of the BNN over the NN is that you can automatically calculate an error associated with your predictions when dealing with data of unknown targets. 
With a BNN, we are now doing Bayesian inference. Let's define our BNN prediction as $\bar{f}(x′|x,t)=∫f(x′,ω)p(ω|x,t)dω$, where $f$ is the NN function, $x'$ are your inputs, $ω$ are the NN parameters, and x,t are the training inputs and targets. This should be compatible with the syntax used by Neal in the links provided by @forecaster. Then we can calculate a standard deviation of the posterior predictive distribution, which I would naively use as an accuracy on the prediction : $\sigma(x′)=\sqrt{∫[f(x′,ω)−\bar{f}(x′|x,t)]^2p(ω|x,t)dω}$
