What are the downsides of bayesian neural networks?

Bayesian neural nets (BNN) are very popular topic. With development of variational approximation it became possible to train such models much faster then with Monte Carlo sampling. BNNs allow such interesting features as natural regularisation and even uncertainty estimation. So, the question is: why haven't we still completely migrated on BNNs?

I can assume that variational inference does not provide enough accuracy. Is it the only reason?

• That's a good question, I believe most people are not aware of it, the math behind is harder, and the implementation examples scarce, if VI improves training I guess more people will be willing to give a try. Apr 22, 2018 at 4:16

4. This is the most important reason that I think why BNN is not adopted universally instead of NN: the uncertainty we get is not as useful as we thought at first glance. Let's take an easy example: say you have two types of customers. Type A will have equal probability of giving you \$40 or \$60, and Type B will have equal probability of giving you \$30 or \$70. They have equal expectation, but larger uncertainty for the type A customer. Assume your BNN works perfectly well to tell one distribution from another. However, the uncertainty here does not matter if you have one million customers of each, because at that time what matters is not the uncertainty of individual customers, but the uncertainty of average, which goes towards zero when your number of customers goes larger according to law of large numbers. Therefore, you really do not need uncertainty in your model most of the time.