What is the difference between neural network, Bayesian network, decision tree and Petri nets, even though they are all graphical models and visually depict cause-effect relationship.
$\begingroup$
$\endgroup$
1
-
3$\begingroup$ Note that there are also Bayesian Neural Networks (just to be confusing), which are basically just a neural network with a Bayesian treatment of parameters, as far as I can see. $\endgroup$– naught101Sep 20, 2016 at 0:39
Add a comment
|
6 Answers
$\begingroup$
$\endgroup$
9
Wow, what a big question! The short version of the answer is that just because you can represent two models using diagrammatically similar visual representations, doesn't mean they are even remotely related structurally, functionally, or philosophically. I'm not familiar with FCM or NF, but I can speak to the other ones a bit.
Bayesian Network
In a Bayesian network, the graph represents the conditional dependencies of different variables in the model. Each node represents a variable, and each directed edge represents a conditional relationship. Essentially, the graphical model is a visualization of the chain rule.
Neural Network
In a neural network, each node is a simulated "neuron". The neuron is essentially on or off, and its activation is determined by a linear combination of the values of each output in the preceding "layer" of the network.
Decision Tree
Let's say we are using a decision tree for classification. The tree essentially provides us with a flowchart describing how we should classify an observation. We start at the root of the tree, and the leaf where we end up determines the classification we predict.
As you can see, these three models really have basically nothing at all to do with each other besides being representable with boxes and arrows.
answered Apr 21, 2014 at 6:30
-
1$\begingroup$ Theory aside, isn't Decision tree also handling dependencies among variables by spitting data? I am new to machine learning field and am more interested in understanding intuitively which algo to use in what scenario. To me it looks like that you would use Bayesian network when you know the exact relationship between variables whereas you would use Decision tree when you guess that some variables may be dependent on others but do not know exactly which ones. Would like to know if this seems correct or if applicability of Bayesian network and decision trees can be explained in more detail. $\endgroup$ Jul 19, 2016 at 6:55
-
$\begingroup$ I really think the answer should point to both differences and similarities to sketch the bigger picture, stating that "these three models really have basically nothing at all to do with each other" is just wrong. Decision Tree and Neural Networks take the same discriminative approach, as compared to the generative approach of BN. While the Other two represent functions, Bayesian Networks represent generalised functions (distributions), ... $\endgroup$– LejafarJan 13, 2018 at 19:34
-
1$\begingroup$ The descriminative vs. generative distinction you are making here is wrong. Neural networks can be used to construct generative models. Consider GANs for example. My answer is sufficient because the question boils down to "these models are represented visually in a similar way, does that mean they are structurally similarly?" and I explained how these graphical representations are encoding very different information. $\endgroup$ Jan 13, 2018 at 22:16
-
1
-
2$\begingroup$ @Lejafar, do not use edits to change the content of someone else's answer. If you disagree with an answer, leave a comment, downvote, or post your own answer. $\endgroup$ Jan 14, 2018 at 20:18
$\begingroup$
$\endgroup$
3
It is easy to show (see Daphne Koller's course) that Logistic Regression is a restricted version of Conditional Random Fields, which are undirected graphical models, while Bayesian Networks are directed graphical models. Then, Logistic Regression could also be viewed as a single layer perceptron. This is the only link (which is very loose) that I think could be drawn between Bayesian Networks and Neural Networks.
I have yet to find a link between the other concepts you asked about.
answered Jan 21, 2016 at 16:34
-
2$\begingroup$ Welcome to the site, & thank you for this contribution. Can you elaborate on how it is easy to see this? At present, this is just a claim that may not be obvious to people. The info may be at the link, but we want this thread to remain informative even after the link goes dead. $\endgroup$ Jan 21, 2016 at 16:42
-
$\begingroup$ Hi & thanks! I'm not sure how fine grained the explanation should be (it takes Daphne at least half an hour to prep the proof) and it's not particularly fresh in my mind either, but the general idea is that the logistic model is a simplified version of the Gibbs Distribution, which in turn is the basis of CRFs. $\endgroup$ Jan 25, 2016 at 15:47
-
1$\begingroup$ What does crf mean? Does crf = conditional random fields? $\endgroup$ Sep 23, 2017 at 15:53
$\begingroup$
$\endgroup$
First we attempt to state the nature of problem attempted to be solved by these methods. If a problem is straightforward, Polynomial or NP Complete we have ready to plug algorithms that could provide with a deterministic answer, by simple recombination of the axioms along logical rules. However, if that is not the case, we would have to rely on a method of reasoning, wherein, we attempt to treat the problem as being heterogeneous and plug it to a network, the nodes being evaluations and the edges being pathways between the components.
In any kind of network based reasoning, we do not reason deductively, by using abstract generalisations and combinations, according to logical rules in a linear flow, but rather work through the problem based on the propagation of reasoning in different directions, such that we solve the problem one node at a time, open to improvements on discovery of new facts concerning any node in the future. Now let us see how each of these techniques approach this method of problem solving in their own way.
Neural Network: The Neural network is a black box, where it is believed (never could be verified from outside the system) that connections among simpleton nodes are formed and emphasised by repeated external reinforcements. It approaches the problem in a Connectionsitic paradigm. The problem is likely solved, but there is little by way of explainability. The neural net now widely used because of its ability to produce quick results, if the problem with explainability is overlooked.
Bayesian Network: The Bayesian Network is a directed acyclic graph, which more like the flowchart, only that the flow chart can have cyclic loops. The Bayesian network unlike the flow chart can have multiple start points. It basically traces the propagation of events across multiple ambiguous points, where the event diverges probabilistically between pathways. Obviously, at any given point in the network, the probability of that node being visited is dependent on the joint probability of the preceding nodes. The Bayesian network is different from the Neural Network in that it is explicit reasoning, even though probabilistic and hence could have multiple stable states based on each step being revisited and modified within legal values, just like an algorithm. It is a robust way to reason probabilistically, but it involves encoding of probabilities, conjecturing the points where randomized actions can happen and hence need more heuristic effort to build.
Decision Trees: The Decision tree is again a network, which is more like a flow chart, which is closer to the Bayesian network than the neural net. Each node has more intelligence than the neural net and the branching can be decided by mathematical or probabilistic evaluations. The decisions are straightforward evaluations based on frequency distributions of likely events, where the decision is probabilistic. However, in Bayesian networks, the decision is based on the distribution of 'evidence' that points to an event having occurred, rather than the direct observation of the event itself.
An Example For instance, if we were to predict the movement of a man-eating tiger across some Himalayan villages that happens to be in the edge of some tiger reserve, we could model it on either approach as follows:
In a decision tree, we would rely on expert estimates whether a tiger would given the choice between open fields or rivers would choose on the latter. In a Bayesian network, We track the tiger by pug marks, but reason in a manner that acknowledges that these pug marks might have been those of some other similar sized tiger routinely patrolling its territory. If we are to use a neural net, we would have to train the model repeatedly using various behavioural peculiarities of the tiger in general, such as its preference to swim, preference of covered areas over open areas, its avoidance of human habitations in order to allow the network to generally reason over the course the tiger might take.
$\begingroup$
$\endgroup$
Excellent answer by @David Marx. I have been wondering, what is the difference between Classification/Regression tree and Bayesian network.One builds on entropy to classify an outcome into classes based on different predictors and the other builds a graphical network using conditional independence and probabilistic parameter estimates.
I feel that the methodology of building the Bayesian network is different compared to the Regression/Decision tree. The algorithm for structural learning, objectives for using the models as well as inferential ability of the models are different.
The score-based and constrained based approach can be understood with some parallels drawn with the information gain criteria in the decision tree families.
$\begingroup$
$\endgroup$
Regarding graphical models, Petri Net formalises a system behaviour; in that it sharply differs from the rest of the mentioned models, all of which relate to how a judgement is formed.
Worth to note that most of the cited names designate quite extensive AI concepts, which often coalesce: for example, you may use a Neural Network to build a decision tree, while the Neural Network itself, as an earlier post discussed, may depend on Bayesian inference.
$\begingroup$
$\endgroup$
Its a good question and I've been asking myself the same. There are more than two kinds of neural network, and it seems the previous answer addressed the competitive type, whereas the Bayesian network seems to have similarities to the feed-forward, back-propagation (FFBP) type, and not the competitive type. In fact, I would say the Bayesian network is a generalisation of the FFBP. So the FFBP is a type of Bayesian network and works in a similar fashion.
