2
$\begingroup$

I have been reading the Deep Learning Book by Ian Goodfellow, and in that, there is a discussion about graphical models like Bayesian belief networks and Markov Random Fields. Here:

One key difference between directed modeling and undirected modeling is that directed models are defined directly in terms of probability distributions from the start, while undirected models are defined more loosely by clique functions that are then converted into probability distributions

Now, what I understand is that MRFs are undirected, represented with the help of clique functions which may not necessarily give valid probabilities which is why we need the partition function. My question then is for MRFs especially: MRFs are graphs that satisfy the Markov assumptions, which are basically a set of conditional independence assumptions, then why cannot we represent them and the graph directly in terms of probability distribution (instead of cliques)?

$\endgroup$

1 Answer 1

2
$\begingroup$

The intuition is that directed graph models, i.e. Bayesian (belief) networks (BN), describe the situation where the joint pdf is written as the product of conditional pdfs, while an MRF describes the situation when the joint pdf is given as a product of factors (which, in the best case, each describe a maximal subset with no conditional independencies). Each has its strengths and weaknesses.

Thus, in the same way, as you can "directly" define a BN from the joint pdf as a product of conditionals, you can "directly" define the MRF from the pdf as a product of factors (and those factors could, of course, also be the conditionals).

E.g., if you have the pdf: $$ p(a, b, c) = p(a)p(b|a)p(c|b), $$ you can directly deduce a BN (not unique!): $$ a\to b \to c $$ and you can directly deduce an MRF (the factors being $p(a)p(b|a)$ and $p(c|b)$): $$ a - b - c. $$

$\endgroup$
1
  • $\begingroup$ Logging back in after a long time. So, just to confirm, you are saying that the factors in the cliques can possibly be conditionals, but not always? Can you also shed light on the strengths and weaknesses of BN and MRF with respect to this $\endgroup$
    – Kunj Mehta
    Sep 12 at 14:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.