6
$\begingroup$

I have a question related to chaper 10.2.3 ("recurrent networks as directed graphical models") of the deep learning book by Bengio et al.

In the chapter, the authors describe how one can interpret an RNN as a graphical model.

However, I still have problems understanding why we interpret RNNs as graphical models at all. What is the benefit of such an interpretation? And do we only interpret RNNs as graphical models or can we apply the idea to all kinds of neural networks?

Improve this question
$\endgroup$
7
  • $\begingroup$ The definition of a graphical model is: "a probabilistic model for which a graph expresses the conditional dependence structure between random variables." As we can draw a dependency graph to represent a NN, it falls in this category of "graphical models". Where does your misunderstanding stems from? Could you precise it so maybe someone can provide a more tailored answer to your concerns :) $\endgroup$
    – Eskapp
    Commented Jan 22, 2017 at 18:06
  • $\begingroup$ I don't understand why we interpret them as graphical models at all. What is the motivation? Is there any benefit when viewing them as graphical models? $\endgroup$
    – Lemon
    Commented Jan 24, 2017 at 15:31
  • $\begingroup$ Let's take the question another way around: "Why do we interpret a Gaussian as a continuous function?". Because, the Gaussian function verifies all the conditions that define what we call "continuous functions" and that theorems, techniques have been developed solely relying upon this set of conditions. It gives us tools that can be used to go forward. Then the community of scientists have defined what "graphical models" are. Tools have been developed that apply to models matching this definition. NN is one of them, it is a graphical model. I don't think there's anything more than this. $\endgroup$
    – Eskapp
    Commented Jan 24, 2017 at 16:06
  • 1
    $\begingroup$ So we can interpret every neural network as a graphical model? Could you be a bit more precise about what kind of tools we can use and what additional benefit they give us compared to the standard definition of a neural network? $\endgroup$
    – Lemon
    Commented Jan 26, 2017 at 13:19
  • 1
    $\begingroup$ Thanks! I will work my way through it. Hopefully it can solve my questions $\endgroup$
    – Lemon
    Commented Jan 28, 2017 at 12:56

1 Answer 1

Reset to default
5
$\begingroup$

As a compilation of my comments on the question:

The definition of a graphical model is: "a probabilistic model for which a graph expresses the conditional dependence structure between random variables." As we can draw a dependency graph to represent a NN, it falls into this category of "graphical models".

About the question, why do we interpret them as graphical models?

Well, we can take the question another way around for illustration purpose:

"Why do we interpret a Gaussian as a continuous function?". Because, the Gaussian function verifies all the conditions that define what we call "continuous functions" and that theorems, techniques have been developed solely relying upon this set of conditions. It gives us tools that can be used to go forward. Then the community of scientists have defined what "graphical models" are. Tools have been developed that apply to models that match this definition. NN is one of them, it is a graphical model. I don't think there's anything more than this.

Then maybe your question is, why did we decide to create a global name for this set of models that we call today "graphical models"? I do not think it was a decision made by people in a voluntarily conscious way. More something that appeared naturally with time and research works. Scientists noticing similarities between several models and reporting them. Slowly, the concept of graphical model was born. I would be happy if other people could share their opinion on this :)

I do not consider myself an expert but I think that NNs are graphical models by definition. I found this very interesting chapter called "Graphical models in a nutshell" that should be able to answer your concerns and more way better than I can do: https://ai.stanford.edu/~koller/Papers/Koller+al:SRL07.pdf

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ link is broken! $\endgroup$
    – Nikola
    Commented Sep 10, 2021 at 9:21
  • 1
    $\begingroup$ Link updated :) $\endgroup$
    – Eskapp
    Commented Sep 13, 2021 at 20:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.