For training a neural network is there any significant difference between using the Sum Squared error or the Mean squared error as the loss function?
According to the theory of optimization calculus, these are equivalent: monotonic transformations like dividing by a constant do not change the $\arg\min$ that you aim to find (that $\arg\min$ gives the network weight and bias values).
However, to do any empirical work, we need to use computers to approximate these optimizations, and the sum and mean do not behave quite the same way. For example, this answer discusses why the mean might be preferred to the sum for numerical reasons when it comes to doing the calculations on a computer like you have to do. Thus, despite the theoretical guarantees of the sum and mean having the same $\arg\min$, the numerical considerations matter for doing applied work.