How to update weights in Bayesian neural network If I understand correctly, in BNN, we compute posterior and this becomes our new prior(new updated weights). But the problem I don't understand is, how do you update new weights, since unlike in deterministic neural network, you don't update point estimate. If I understand correctly, you need to apply new mu and sigma parameters on weights (if gaussian is the case).
I would appreciate your reply
 A: Only for the trivial problems you can apply Bayes theorem directly to calculate the posterior distributions. For almost all real-life statistical problems we use algorithms for approximating the posterior distribution. Currently the most popular approach to this is using Markov Chain Monte Carlo (see questions tagged as mcmc), for example the NUTS sampler (Hoffman and Gellman, 2011). Since for Bayesian neural networks those algorithms may be inefficient because of the huge number of parameters, in such cases people often use variational inference (see variational-bayes), with algorithms like ADVI (Kucukelbir et al, 2016) to estimate the posterior distributions. This is dynamically changing field, with a lot of ongoing research on this subject, and many more algorithms were proposed in the literature, so it is hard to summarise it in Q&A site answer. Moreover, it is plausible that the methods that are popular in this field today, would not be that popular in a year or two, since we would find better algorithms. I would encourage you to google for "bayesian neural networks", since this would reveal thousands of relevant search results including academic papers, books, online tutorials, recordings of conference talks etc. on this subject.
