4
$\begingroup$

I have a question related to chaper 10.2.3 ("recurrent networks as directed graphical models") of the deep learning book by Bengio et al.

In the chapter, the authors describe how one can interpret an RNN as a graphical model.

However, I still have problems understanding why we interpret RNNs as graphical models at all. What is the benefit of such an interpretation? And do we only interpret RNNs as graphical models or can we apply the idea to all kinds of neural networks?

$\endgroup$
  • $\begingroup$ The definition of a graphical model is: "a probabilistic model for which a graph expresses the conditional dependence structure between random variables." As we can draw a dependency graph to represent a NN, it falls in this category of "graphical models". Where does your misunderstanding stems from? Could you precise it so maybe someone can provide a more tailored answer to your concerns :) $\endgroup$ – Eskapp Jan 22 '17 at 18:06
  • $\begingroup$ I don't understand why we interpret them as graphical models at all. What is the motivation? Is there any benefit when viewing them as graphical models? $\endgroup$ – Lemon Jan 24 '17 at 15:31
  • $\begingroup$ Let's take the question another way around: "Why do we interpret a Gaussian as a continuous function?". Because, the Gaussian function verifies all the conditions that define what we call "continuous functions" and that theorems, techniques have been developed solely relying upon this set of conditions. It gives us tools that can be used to go forward. Then the community of scientists have defined what "graphical models" are. Tools have been developed that apply to models matching this definition. NN is one of them, it is a graphical model. I don't think there's anything more than this. $\endgroup$ – Eskapp Jan 24 '17 at 16:06
  • $\begingroup$ Then maybe your question is, why did we decide to create a global name for this set of models that we call today "graphical models"? I do not think it was a decision made by people in a voluntarily conscious way. More something that appeared naturally with time and research works. Scientists noticing similarities between several models and reporting them. Slowly, the concept of graphical model was born.I would be happy if other people could share their opinion on this :) $\endgroup$ – Eskapp Jan 24 '17 at 16:10
  • 1
    $\begingroup$ Thanks! I will work my way through it. Hopefully it can solve my questions $\endgroup$ – Lemon Jan 28 '17 at 12:56
1
$\begingroup$

As a compilation of my comments on the question:

The definition of a graphical model is: "a probabilistic model for which a graph expresses the conditional dependence structure between random variables." As we can draw a dependency graph to represent a NN, it falls into this category of "graphical models".

About the question, why do we interpret them as graphical models?

Well, we can take the question another way around for illustration purpose:

"Why do we interpret a Gaussian as a continuous function?". Because, the Gaussian function verifies all the conditions that define what we call "continuous functions" and that theorems, techniques have been developed solely relying upon this set of conditions. It gives us tools that can be used to go forward. Then the community of scientists have defined what "graphical models" are. Tools have been developed that apply to models that match this definition. NN is one of them, it is a graphical model. I don't think there's anything more than this.

Then maybe your question is, why did we decide to create a global name for this set of models that we call today "graphical models"? I do not think it was a decision made by people in a voluntarily conscious way. More something that appeared naturally with time and research works. Scientists noticing similarities between several models and reporting them. Slowly, the concept of graphical model was born. I would be happy if other people could share their opinion on this :)

I do not consider myself an expert but I think that NNs are graphical models by definition. I found this very interesting chapter called "Graphical models in a nutshell" that should be able to answer your concerns and more way better than I can do: https://homes.cs.washington.edu/~taskar/pubs/gms-srl07.pdf

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.