Having looked through the internet and the paper, I find Bayes by Backprop very unaccesible for my intermediate understanding of variational inference.
Most online guides also lack some explaining like this (https://www.nitarshan.com/bayes-by-backprop/) explains the entire ELBO idea, but then doesn't elaborate on why it is not just a variational autoencoder.
What is the difference between a variational autoencoder and Bayes By Backprop type neural net? Is it just effectively "encoder" of the VAE but regressed against y? If that is the case, how is that different from a Mixture Density type neural network?
I could look into the code, but my approach is to understand them separately so that I can stay critical about the implementations - and everyone can post any code so there is a reason to be critical.