Explanation for MSE formula for vector comparison with "Euclidean distance"?

I study the loss functions for regression tasks with artificial neural networks. In case of evaluating loss with Mean Squared Error for multidimensional outputs I read the following usual formula which is straightforward for me (N is the number of samples, M is the output dimensionality):

$\L(W,b)=\frac{1}{N}\sum_{i=1}^{N}\frac{1}{M}\sum_{j=1}^{M}(\hat{y}_{ij}-y_{ij})^{2}$

However, I also confronted with a slightly different format:

$\L(W,b)=\frac{1}{2N}\sum_{i=1}^{N}\sum_{j=1}^{M}(\hat{y}_{ij}-y_{ij})^{2}$

The accompanying explanation says:

"You’ll recognize the inner sigma in the preceding equation as the square of the Euclidean distance. In fact, the MSE is sometimes referred to by these terms. Note that N and M are constants. So, consider these as simple scaling factors that you can account for in other ways (like by scaling the learning rate). In a lot of use cases M is dropped and a division by two is added for mathematical convenience (which will become clearer in the context of its gradient in backpropagation)."

I cannot follow this explanation (and neither the second formula).

• What is the connection between the formula and the Euclidean distance?
• How does the fact that "N and M are constants" explain anything?
• How scaling factors come into the picture?
• What is the "mathematical convenience" here and how does it become clearer in backpropagation? (I'm more or less familiar with backprop).
• Where did you find those definitions? Commented Nov 3, 2021 at 21:16

What is the connection between the formula and the Euclidean distance?

Consider the formula of the Euclidean Distance between $\hat{y}$ and $y$ when they have same dimensionality:

$D = \sqrt{\sum_{i=0}^n (\hat{y}_{i} - y_{i})^2 }$

so the square is:

$D^2 = {\sum_{i=0}^n (\hat{y}_{i} - y_{i})^2 }$

that is very close to you formula except for the factors $N$ and $M$.

How does the fact that "N and M are constants" explain anything? How scaling factors come into the picture?

Basically, the fact that $N$ and $M$ are costants means that they you can see them as scaling factors. Indeed you will divide your amount always for the same quantity.

What is the "mathematical convenience" here and how does it become clearer in backpropagation? (I'm more or less familiar with backprop).

It's just for a matter of convenience, because in the backprogation you need to compute the derivative of your squared term. So if your loss function (bias omitted for simplicity) is:

$L = \frac{1}{2N} \sum_{i=0}^n (\hat{y}_{ij} - y_{ij})^2$

when apply the power rule on it for getting the derivative wrt the weights, you obtain:

$\frac{\partial L}{\partial W} = \frac{1}{N} \sum_{i=0}^n (\hat{y}_{ij} -y_{ij})$

So the constant $2$ has disappear from the equation.