0
$\begingroup$

It is common to draw a neural network as a "web" of neurons and connections, such as the "web" below of a multilayer perception that has input neurons in white, hidden neurons in black, and the output neuron in blue.

MLP web

It is also possible to do this for a convolutional neural network, such as the drawing below that shows a 3x3 image being mapped to four neurons.

CNN web

I find these visualizations, particularly for a convolutional neural network, quite useful in seeing these deep learning models as real regression models instead of some kind of import tensorflow as abracadabra software magic. For the convolutional neural network, I can see that the feature corresponding to the pixel in the top left is multiplied by some value (the red weight), and then the red, blue, grey, and purple products are added together in the hidden neurons before being transformed by some activation function. If need be, I can turn the web into a regression equation. Sure, real image recognition work might zero-pad the image and would have multiple convolution layers with some max pooling, but all of that fits in this framework.

Enter graph neural networks.

I cannot figure out how to draw the architecture of a graph neural network. Especially frustrating is the fact that each node in a graph can have varying numbers of neighbors, yet I think of a regression as a function that maps a fixed-dimension space to an output space.

I've gone through some material on graph neural networks and think I'm starting to see how they act as feature extraction laters to generate that fixed-dimension feature space. You start out with fixed-dimension feature vectors for the neurons and then tweak those values to reflect the characteristics of the node's neighbors (and their neighbors' neighbors, and their neighbors' neighbors' neighbors, etc).

However, is there a way to draw a simple yet illustrative example of a graph neural network architecture the way I have done for a convolutional neural network?

$\endgroup$
2
  • $\begingroup$ I your CNN image are the four neurons the output neurons or are they just the new neurons after a single convolution. In mind mind I would stack two graphs, moved a little bit apart. And then you can use colored lines to map from the neighboring nodes of the lower graph to central atom, that they are neighboring on the graph stacked on top. $\endgroup$
    – Janosch
    Sep 13, 2023 at 14:31
  • $\begingroup$ @Janosch I don’t think it matters if that’s the entirety of the network or just one of the convolution layers, though I have always assumed there would be subsequent layers of some kind after that stack of four. $\endgroup$
    – Dave
    Sep 13, 2023 at 15:07

1 Answer 1

0
+50
$\begingroup$

Graph Neural Networks vary greatly on how it connects the graph structure to neural networks layers, but as I understand it usually works very much like a CNN but instead of images we have graphs and node neighborhood is accounted for on layer connections and the original structure is maintained until a certain point where some type of transformation is performed (like flatten on CNNs).

Tried to draw a simple representation of it like you did with other architectures: enter image description here

but just draw the connections for two nodes because it would be confuse with all the connections and my lack of drawing skills.

$\endgroup$
2
  • $\begingroup$ $1)$ What do the colors mean? $//$ $2)$ In which direction do the connections point? $//$ $3)$ You took the time to answer, so despite questions remaining on my end, $+50$. $\endgroup$
    – Dave
    Nov 9, 2023 at 15:53
  • $\begingroup$ Hello! I draw the "network convolution" for two nodes (the two kind of central ones that are getting connections on the right network) from left to right like left is network before the nn layer and right is a resulting network after the application of the layer. For the first node that convolution is represented , I used blue for the edge that uses features from the node itself and orange for the features of neighbors. For the second node, light blue for the node with itself edge and purple for the neighbors edges. $\endgroup$ Nov 9, 2023 at 18:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.