The form for MSE for $N$ data points with scalar values $Y=[Y_1,Y_2,...,Y_N]$ is given by the formula: $$ MSE = \frac{1}{N} \sum_{i=1}^N (Y_i - \hat{Y}_i)^2 $$

How I see it, $ d_i = Y_i - \hat{Y}_i$, where $d_i$ is the Euclidean distance between the actual and predicted values for the $i^{th}$ data point.

Thus, extending this to higher dimensions, say $D$ dimensions, $Z=[\vec{Z_1},\vec{Z_1},...,\vec{Z_N}]$. Thus, the MSE should be: $$ MSE = \frac{1}{N} \sum_{i=1}^N d_i^2 = \frac{1}{N} \sum_{i=1}^N \|Z_i - \hat{Z}_i\|^2 = \frac{1}{N} \sum_{i=1}^N \sum_{j=1}^D (Z_{ij} - \hat{Z}_{ij})^2 $$

However, although I did not see any direct result which mentions this, it seems most implementations of MSE use a different formula (not too different from what I thought above): $$MSE' = \frac{1}{N} \sum_{i=1}^N \frac{1}{D} \sum_{j=1}^D (Z_{ij} - \hat{Z}_{ij})^2$$

  1. Is this $MSE'$ the correct form? If the MSE should provide the Mean Squared Error, where the error is measured by the Euclidean distance between the points, then why is this averaging over $D$ too?
  2. I do know that it doesn't make too much of a difference (a constant factor) if we use one of the measures consistently, but which one is standard? Is there a unique definition of MSE in these cases?
  • $\begingroup$ Related? $\endgroup$
    – Dave
    Commented Apr 12, 2023 at 19:44
  • $\begingroup$ I need some info about which one is the standard notion of MSE in this context of higher dimensions. From the linked answer, it seems my intuition is not incorrect, but why do the standard implementations have it the other way? $\endgroup$
    – Aditya
    Commented Apr 12, 2023 at 20:47
  • $\begingroup$ Your formula for "MSE" is the mean of a kind of total error across all $D$ dimensions. (It's rather a strange kind, since it averages across the dimensions, but I will suppose there are setting where it's meaningful. It's related to the mean squared error term in ANOVA, for instance.) Maybe that's what you need; maybe not: you have to determine what is appropriate for your analysis. $\endgroup$
    – whuber
    Commented Apr 12, 2023 at 21:10


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.