# (How) can model parameters be learnt using MCMC?

I get stuck by part (b) of figure 4 in this paper: Hands-on Bayesian Neural Networks - a Tutorial for Deep Learning Users.

In my understanding, inference algorithms like MCMC are not for training because the posterior of a variable should be fixed before being sampled.

In that figure, MC-Dropout is also grouped into the learning stage. But I thought it is only used in the validation step.

Can inference algorithms like HMC and MC-Dropout be applied for parameter learning using data? Or am I missing something? It would be highly appreciated if anyone could help me out. Thanks in advance.

I learned from this tutorial that No-U-Turn Sampler(NUTS) is used to sample and get the distributions of four variables, but I don't know how the sample function works, what is going on under the hood? I know the mechanism of MCMC, but what confused me is that: could that process be the same if the network is much more complex than linear regression? For instance, a deep neural network?

with basic_model:
# draw 500 posterior samples
trace = pm.sample(500, return_inferencedata=False)


I have a deep-learning-stochastic-gradient-descent mindset and cannot get my head around the sampling mechanism(sample to get the variable distribution?). How are the variables "trained" by data(no matter how expensive)? Or they can only be analytically calculated/derived and then be sampled to plot their distribution?

In my understanding MC-Dropout is for testing only. Arguably, though, its a final step in inference, you do it once right at the very end to generate an ensemble. So, it can be included in inference in a backhand way as the final step.

MCMC and variational methods are for inference. The "posterior" of a variable is an abstract theoretical concent that exists in principle, and with deep neural networks we never get to represent it fully. MCMC and variational methods are alternative methods for (approximately) sampling from the posterior or approximating the posterior.

Parameter learning is not really something that fits neatly into the Bayesian framework. The best you can do is approximate, sample, estimate. But the results of typical deep learning, because its always stochastic, can be viewed as a sample from the posterior. For this reason, a very simple approach for ensembling is to train 5 networks independently (with different batch presentations). But this is 5 times slower than normal, and why MC-dropout was invented.

• I wonder if this relates to the mixture of conjugate priors? May 30 at 6:25

MCMC algorithms are used to sample complex distributions, but a lot of point parameter estimation formulas can be reduced to a sampling problem.

For example, the integral to compute the expected value of a random variable can be estimated using Monte-Carlo, which reduce the computation of the expected value to a sampling problem.

The same for a maximum likelihood estimation: if you sample a random variable long enough, you can build an histogram and thus get an estimation of the maximum likelihood value of the variable.

Note that, however, for most point estimation problems, generic MCMC methods are likely to be very slow compared to more efficient procedures. For example you would not use HMC to get a point estimate of a neural network parameters. A variant of the gradient descent algorithms would be much faster. It is important to note thus, that the Stochastic Gardient Descent algorithm, which is the goto method for neural network training today, can be considered to be a Markov Chain algorithm.

(Also, The name training in the figure is an approximation. What you are performing at this step is inference, but it correspond to the training phase when compared to a point-estimate model workflow).