2
$\begingroup$

Let's say I train a model and it has an RMSE of 2.5.

Does this mean, that on average, my prediction will be 2.5 away from the true value? Or does some scaling need to be done in oreder to get this value's magnitute to be in line with the target variable's magnitude?

$\endgroup$

1 Answer 1

3
$\begingroup$

Does this mean, that on average, my prediction will be 2.5 away from the true value?

No. What you described is the definition of the bias of the estimator.

RMSE is the square root of the mean of the squared deviation of the prediction from the predicted value. So it tells you how wrong your model is on average, but the same applies to any other error metric.

If your prediction for all the values was the global arithmetic average of the data, RMSE would be equal to the standard deviation of residuals. So a reasonable model should aim at a value lower than this.

You may also want to read more about that do estimate the bounds for the predictions.

$\endgroup$
1
  • $\begingroup$ One detail that's missing but is actually relevant to what RMSE means is whether the RMSE was computed on the training data, a holdout dataset or a test set. $\endgroup$
    – dipetkov
    Commented Apr 10, 2023 at 17:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.