As a compilation of my comments on the question:
The definition of a graphical model is: "a probabilistic model for which a graph expresses the conditional dependence structure between random variables." As we can draw a dependency graph to represent a NN, it falls into this category of "graphical models".
About the question, why do we interpret them as graphical models?
Well, we can take the question another way around for illustration purpose:
"Why do we interpret a Gaussian as a continuous function?". Because, the Gaussian function verifies all the conditions that define what we call "continuous functions" and that theorems, techniques have been developed solely relying upon this set of conditions. It gives us tools that can be used to go forward. Then the community of scientists have defined what "graphical models" are. Tools have been developed that apply to models that match this definition. NN is one of them, it is a graphical model. I don't think there's anything more than this.
Then maybe your question is, why did we decide to create a global name for this set of models that we call today "graphical models"? I do not think it was a decision made by people in a voluntarily conscious way. More something that appeared naturally with time and research works. Scientists noticing similarities between several models and reporting them. Slowly, the concept of graphical model was born. I would be happy if other people could share their opinion on this :)
I do not consider myself an expert but I think that NNs are graphical models by definition. I found this very interesting chapter called "Graphical models in a nutshell" that should be able to answer your concerns and more way better than I can do: https://homes.cs.washington.edu/~taskar/pubs/gms-srl07.pdf