What's a reasonable mean squared error (or RMSE)?

For a given regression dataset, with target $$Y \in \mathbb{R}$$, where the training set examples have $$\mathrm{std}(Y)$$ and $$\mathrm{mean}(Y)$$, what is a mean squared error that makes the result obvious or not interesting? What would be a "reasonable MSE"? (all in relation to the standard deviation and the mean of $$Y$$).

For example, it doesn't make much sense for such MSE to be larger than the standard deviation squared (variance), because choosing a constant target to be the mean would give such error?

• Well, one option would be to, as you figured out, to use a constant - such as mean - as a simple forecast, and compare it to that. This is what $R^2$ does, compares your model to a constant forecast model (plus another constant). Jun 23 at 12:31
• Related, though not identical: stats.stackexchange.com/questions/414349/…
– mkt
Jun 23 at 12:33

You would want the MSE to be as small as possible compared to the variance. A standard measure for this is the coefficient of determination, also known as "R squared": $$R^2 = 1 - \frac{MSE}{var(Y)}.$$ But what is "reasonable" very much depends on your situation.
As you can see, this doesn't depend on $$mean(Y)$$, as would e.g. the relative error. The coefficient of determination $$R^2$$ tells you how good your model fit is compared to other models. The relative error, on the other hand, is often used when considering the usability of your approximation. E.g., if you know that the residuals are of the size of the means, you might have to reconsider your experimental setup.
• thank you, I now understand $R^2$. Jun 23 at 13:41