Where to learn probabilistic deep learning/baysian methods for machine learning I have completed the machine learning course and deep learning specialization by Andrew Ng on Coursera, and now learning TensorFlow 2 for Deep Learning Specialization by the imperial college of London, which is a course that teaches you how to use TensorFlow. However, when I head to the third course, it talks about Probabilistic Deep Learning. I do not have any idea of what bayesian is. For prerequisite, it recommends this course:
https://www.coursera.org/learn/bayesian-methods-in-machine-learning. I have watched the first few videos, but the way it teaches is not for me. I am wondering, are there any other sources of learning this topic. I know basic probability theory, both univariate and multivariate distributions, variance, MGFs, expectations, etc.
 A: It depends on how advanced you are.  I would recommend William Bolstad's "Introduction to Bayesian Statistics" and his "Understanding Computational Bayesian Statistics."
In Pearson and Neyman's method of statistics with its minimum variance unbiased estimator, uniformly most powerful tests, and so forth, all information comes from the data and the data alone.  For example, imagine you calculate a confidence interval for calories and get the interval [-10,90].  From domain knowledge, you know that negative calories cannot exist.
That is domain knowledge.  It is not in the data.  Bayesian methods require you to put outside information into the methodology.  Pearson and Neyman's methods don't allow outside information.
Bayesian methodologies differ from Frequentist methods in that data is not considered random.  Parameters are random.  Randomness does not imply chance, it implies uncertainty.
There is a good discussion of Frequentist confidence intervals versus Bayesian credible intervals at here.
Bayesian methods start with a distribution of the knowledge you have about the problem seen outside the collection of the data.  That distribution is multiplied by the likelihood function and normalized.  That gives you a new distribution.
The distributions quantify your uncertainty about parameters.
If you need a theoretical grounding, pick up E.T. Jaynes "Probability Theory: The Language of Science."  Also, if you need a grounding in decisions, grab Christian Robert's "The Bayesian Choice."  Both are rigorous.  Neither are for beginners.
There are three primary axiomatizations of Bayesian probability theory.  Jaynes provides one of them.  The other two are by Leonard Jimmie Savage and Bruno de Finetti.  The axioms do not match Kolmogorov's.  Because of that, Bayesian and Frequency based methods do not produce the same results.  Jaynes' construction follows from Richard Cox's axioms.
Cox's axioms are built on the theory of logic.  Savage's are built on preference theory and utility theory.  De Finetti's axioms are built on gambling.  They contrast with Kolmogorov's method which is derived from the measurement of sets and the basis of the probability theory you know.
Cox's perspective would be "what is the probability a statement from logic is true?"
De Finetti's perspective might be "how much money would I gamble on some assertion and at what odds?"  Further, "how would those odds change as I garnered new information?"
Savage's perspective is a bit challenging because it is grounded in utility theory.  It begins with $x\succ{y}$ as its first axiom. He might ask "what is your personal estimate of $\theta?$
Give yourself some time to get adjusted to the way of thinking.  If you follow the link above, you will see that the calculations are orthogonal to the statistics you are used to.  It produces different intervals for the same phenomenon in that link.
Any system that begins with logic, gambling, or your satisfaction won't look like an impersonal math built on sets and optimal estimators and tests.
A: Machine Learning: A Bayesian and Optimization Perspective (2nd Edition)
by Sergios Theodoridis seems to squarely address your interests:

*

*it builds from topics you know about probability theory to topics in machine learning


*it covers a wide range of machine learning topics


*it motivates machine learning topics from the perspective of Bayesian reasoning
