# What is MCRMSE (mean columnwise root mean squared error)?

The MCRMSE evaluation metric was used in the Kaggle Competitions Africa Soil Property Prediction Challenge(6 years ago) and OpenVaccine: COVID-19 mRNA Vaccine Degradation Prediction(On-going) competitions. There was no topic regarding MCRMSE (mean columnwise root mean squared error) on the internet.

AFAIK

Root Mean Squared Error - RMSE is the square root of the mean/average of the square of all of the error.

The use of RMSE is very common and it makes an excellent general purpose error metric for numerical predictions. Compared to the similar Mean Absolute Error, RMSE amplifies and severely punishes large errors. The formula for calculating RMSE is:

$$\mathrm{RMSE} = \sqrt{\frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2}$$

Mean Columnwise Root Mean Squared Error - MCRMSE

$$MCRMSE = \frac{1}{m}\sum_{j=1}^{m}\sqrt{\frac{1}{n}\sum_{i=1}^{n}(y_ {ij}-\hat{y}_{ij})^2}$$

or

$$MCRMSE = \frac{1}{m}\sum_{j=1}^{m}RMSE_j$$

where:

$$m$$ - number of predicted variables,

$$n$$ - number of test samples,

$$y_{ij}$$ - $$i$$-th actual value of $$j$$​-th variable,

$$y_{ij}$$ - $$i$$-th predicted value of $$j$$-th variable

I would like to understand What is MCRMSE? When to use??

When would one use MCRMSE over RMSE?

Normally, we can calculate RMSE to get a single-number evaluation metric for our prediction, but if we are predicting multiple values at once$$-$$in the case of the OpenVaccine competition, we need to predict degradation rates under multiple conditions$$-$$we would get multiple different RMSE values, one for each column.
• This is the correct answer (+1) for why they use this metric. I see a few reasons not to like it. If one output had much higher variability, then you could win the competition by crushing that one at the expense of the others. (Then again, it might be reason able to prioritize certain predictions, but we must watch out for this.) Second, the more typical way to examine this sort of multivariate regression would be with $L_2$ distance (maybe Mahalanobis) in the output space, which, in some sense, has a bit easier of an interpretation. – Dave Sep 13 '20 at 15:10
• Related to that second criticism, this seems like a hybrid of $L_1$ and $L_2$ metrics, harshly penalizing large errors within an model but then mildly punishing a model that, overall, is poor. Perhaps that is desirable, but we should be mindful of that feature of the metric. – Dave Sep 13 '20 at 15:23