Nodes' attribute scaling/normalization before graph embedding learning - GNN? In a node classification setting, is it require to normalize/scale graph node attributes before learning node embeddings using graph neural networks? Why?
 A: It is strongly advised to scale (subtract mean, divide by standard deviation) your input. It almost never hurts but can help a lot with speed and estimation performance of your GNN.
As to why this helps: The standard explanation for why this helps in GNNs, or more generally in any neural network, can be found e.g. here, which gives a pretty clear explanation.
In a nutshell: Any type of gradient descent optimization algorithm has to step into a "valley" of the "error surface", to find the minimum in this valley. This process is slowed down a lot if this valley is very elongated as opposed to nicely round (isotropic). And it is not too hard to see that this elongation happens when the features have very different sizes.
So, in general, if you have an optimization problem with a "quasi-linear" model (like eg. NNs) and you use as algorithm one from the family of gradient descent techniques, you should definitely check whether scaling your inputs improves your optimization. All those conditions are fulfilled when learning node embeddings with GNNS, so you should give it a try.
Having said that, there are situations when scaling might not be a good idea. Take e.g. a set of three two dimensional points $a, b, c, \in \mathbb{R}^2$:
$$
\begin{align}
a &= (-1, 0)\\
b &= (1, 0) \\
c &= (0, 100)
\end{align}
$$
Clearly, the two points $a$ and $b$ cluster at zero while the point $c$ is an anomaly, far away. Unfortunately, after scaling they will all have roughly the same distance. So, if the relation of distances between points of your data is relevant for your problem (e.g. clustering, anomaly detection, ...), you should think twice before scaling.
A: If you think about it: A Graph neural Network used for Node Classification is really similar to a regular neural network. In a regular Neural Network the activation of the last hidden layer will have the dimensions n x f where n is the batchsize and f the feature size (or the output size of that layer).
In the final step we want to make a prediction for each vector within the batch. So for each [i,:]. The same applies to GNN here after the convolutions (or message passing), the activations will also be of size n x f. Here n is not the batchsize but the number of nodes in the Graph. And you want to make for some of the predictions.
So scaling for GNNs is important for the same reason as it is important for other NNs. Large values can make the training unstable.
Generally all gradient-based and distance-based approaches require scaling of variables.
