I'm trying to understand message passing algorithms, especially for the specific application of performing conditioning in a Bayesian network. My question is wether there is a mathematically precise reference on these kinds of method.

By this I mean a text in which the algorithms are specified formally. The literature on this topic has a general tendency to spend a lot of time on informal motivation and examples but always seems to stop short of giving a complete formal description, leaving the reader to fill in all such details by themself. The reader is expected to see that certain terms in an equation can be seen as messages passed from one node to another, but the term "message" itself never seems to be formally defined.

I can see why this would be helpful for a lot of people, but it doesn't suit my reading style at all - I find it very difficult to follow the informal motivation if I don't already have a concrete idea of what it's trying to show. So I'm looking for something that, as much as possible, takes a definition-theorem-proof approach to these algorithms - the kind of text that gives definitions first and proofs afterwards, instead of introducing concepts and deriving them at the same time. Does something like this exist?

Ideally, I'm hoping for something that explains both variational message passing and expectation propagation. I'm hoping for something that can give the 'big picture' of how these algorithms relate to each other, why they work in general, and exactly which inference problems are solved by each algorithm, rather than an introduction to just one specific algorithm.


1 Answer 1


Graphical Models, Exponential Families, and Variational Inference by Martin Wainwright and Michael I. Jordan might fit the bill. Jordan is a very distinguished mathematician, and this book is intended to explain the underlying mathematical unity and variational justification for a host of related algorithms, including message-passing and expectation propagation.

  • 1
    $\begingroup$ Thank you, that looks great! It might take me a while to get around to looking at it, but if it helps me I will accept this answer. $\endgroup$
    – N. Virgo
    Commented Aug 3, 2022 at 0:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.