Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Why are directed graphical models called Bayesian,while those undirected not?

Does that mean that Bayesian analysis is involved in the directed ones, and not in those undirected?

share|improve this question
add comment

3 Answers 3

Bayes networks can be represented graphically by directed acyclic graphs, but not all such graphs are Bayesian. The representation is useful for Bayesian analysis because it illustrates the states of a system as well as each associated transition probabilities. Undirected graphs cannot assign a specific transition probability because the transition direction is ambiguous. A more general case is Markov networks which can handle cyclic and undirected networks, provided the nodes have Markov properties.

share|improve this answer
    
Thanks! the transition between any two random variables is always unidirectional in Bayesian, i.e. we can only talk about transition from one random variable to the other, not from the second to the first? –  Tim Mar 29 at 18:29
    
I wouldn't describe it as transition probability unless you are talking about dynamic models using time. It is more about joint-distributions among a set of variables and how they factor. Some factorizations can be described with a DAG. Some cannot but all can be described by an undirected model. –  Dave31415 Mar 29 at 18:43
    
A DAG has an ordering. You talk about parents and children. With undirected, you just talk about cliques. Normally you just convert it to the equivalent undirected model by moralizing the graph anyway. –  Dave31415 Mar 29 at 18:47
add comment

The point about DAGs is that they reflect the factorization of the density of probability. It does not necessarily have something to do with transitions. Maybe more interestingly (and used in many applications in that sense) has to do with causality. That is, observing some events can have several causes. How can I infer which cause is the most likely?

The fact that DAG reflects how the density of probability factorizes, means that it captures the independence relationships, that is, it allows to reason about the data. Based on the graph structure, is possible to derive efficient algorithms for inference.

But going back to your original question, undirected models are also Bayesian. A typical case are Markov Random Fields (MRF) in image processing. See for example "Bayesian Methods and Markov Random Fields" by Mario A. T. Figueiredo. The idea is that the pixel values (the observations) are the consequences of a cause (which is defined by the application: segmentation, object detection, and so on).

In this indirected models you do not reason in terms of causality, but make inference from relationships among observations and local observations. In undirected models one defines the way the observations interact with each other, and that allows you to solve big combinatorial problems in an efficient manner. You may take a look at the paper: "Markov Random Field modeling, inference & learning in computer vision & image understanding: A survey", by Wang et al. or the wikipedia site on MRFs.

share|improve this answer
add comment

You can (and should) take a Bayesian probabilistic approach with either in the sense of putting prior probabilities on node values. It is not a good name because they are not more Bayesian than undirected models. I think the name just comes from the fact that DAGs were used in expert systems and people applied Bayesian techniques to them and called them Bayesian networks and the name stuck. They should really be called directed and undirected graphical models. Most machine learning techniques are based on probability theory. I don't know of any of those that could be called non-Bayesian.

share|improve this answer
    
I think a distinction should be made between being Bayesian and using methods inspired by what a Bayesian would do under a particular prior and evaluating them via some non-Bayesian criteria like CV. Methods themselves aren't Bayesian or non-Bayesian, it is our analyses. –  guy Mar 29 at 22:06
    
Personally, I don't believe that you should do anything non-Bayesian while you're working within probability. But sometimes you have to leave probability theory behind and do practical things that you can't quite justify with Bayesian probability theory. –  Dave31415 Mar 29 at 22:16
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.