Why are directed graphical models called Bayesian,while those undirected not?
Does that mean that Bayesian analysis is involved in the directed ones, and not in those undirected?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ My impression from Koller/Friedman's book is that the reason is because Bayesian networks are a generalization of something called the 'Naive Bayes model'. (This admittedly leads to the question of why Naive Bayes models are called that, which I don't know.) Also Judea Pearl, who did a lot of work popularizing Bayesian networks, has a really strong (subjective) Bayesian interpretation of them, hence also, e.g., the name 'belief propagation' algorithm, also coined by him. Both the Naive Bayes and Bayesian networks were (and are) used in creating 'expert belief systems'. $\endgroup$– Chill2MachtCommented Dec 10, 2017 at 16:20
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
The point about DAGs is that they reflect the factorization of the density of probability. It does not necessarily have something to do with transitions. Maybe more interestingly (and used in many applications in that sense) has to do with causality. That is, observing some events can have several causes. How can I infer which cause is the most likely?
The fact that DAG reflects how the density of probability factorizes, means that it captures the independence relationships, that is, it allows to reason about the data. Based on the graph structure, is possible to derive efficient algorithms for inference.
But going back to your original question, undirected models are also Bayesian. A typical case are Markov Random Fields (MRF) in image processing. See for example "Bayesian Methods and Markov Random Fields" by Mario A. T. Figueiredo. The idea is that the pixel values (the observations) are the consequences of a cause (which is defined by the application: segmentation, object detection, and so on).
In this indirected models you do not reason in terms of causality, but make inference from relationships among observations and local observations. In undirected models one defines the way the observations interact with each other, and that allows you to solve big combinatorial problems in an efficient manner. You may take a look at the paper: "Markov Random Field modeling, inference & learning in computer vision & image understanding: A survey", by Wang et al. or the wikipedia site on MRFs.
$\begingroup$
$\endgroup$
3
Bayes networks can be represented graphically by directed acyclic graphs, but not all such graphs are Bayesian. The representation is useful for Bayesian analysis because it illustrates the states of a system as well as each associated transition probabilities. Undirected graphs cannot assign a specific transition probability because the transition direction is ambiguous. A more general case is Markov networks which can handle cyclic and undirected networks, provided the nodes have Markov properties.
-
$\begingroup$ Thanks! the transition between any two random variables is always unidirectional in Bayesian, i.e. we can only talk about transition from one random variable to the other, not from the second to the first? $\endgroup$– TimCommented Mar 29, 2014 at 18:29
-
$\begingroup$ I wouldn't describe it as transition probability unless you are talking about dynamic models using time. It is more about joint-distributions among a set of variables and how they factor. Some factorizations can be described with a DAG. Some cannot but all can be described by an undirected model. $\endgroup$ Commented Mar 29, 2014 at 18:43
-
$\begingroup$ A DAG has an ordering. You talk about parents and children. With undirected, you just talk about cliques. Normally you just convert it to the equivalent undirected model by moralizing the graph anyway. $\endgroup$ Commented Mar 29, 2014 at 18:47
$\begingroup$
$\endgroup$
2
You can (and should) take a Bayesian probabilistic approach with either in the sense of putting prior probabilities on node values. It is not a good name because they are not more Bayesian than undirected models. I think the name just comes from the fact that DAGs were used in expert systems and people applied Bayesian techniques to them and called them Bayesian networks and the name stuck. They should really be called directed and undirected graphical models. Most machine learning techniques are based on probability theory. I don't know of any of those that could be called non-Bayesian.
-
$\begingroup$ I think a distinction should be made between being Bayesian and using methods inspired by what a Bayesian would do under a particular prior and evaluating them via some non-Bayesian criteria like CV. Methods themselves aren't Bayesian or non-Bayesian, it is our analyses. $\endgroup$– guyCommented Mar 29, 2014 at 22:06
-
$\begingroup$ Personally, I don't believe that you should do anything non-Bayesian while you're working within probability. But sometimes you have to leave probability theory behind and do practical things that you can't quite justify with Bayesian probability theory. $\endgroup$ Commented Mar 29, 2014 at 22:16