I am working on a data set that has 930 rows and 5 columns. I'm implementing KNN Regression with bootstrapping and my RMSE is almost 400 (depending on different cross validated outputs). I read this post and one of the answers mentioned that if the training and test RMSE are similar, then the model is good; any my training error is close to 400 too. I've tried searching online for reliable sources to know if this high an error rate is acceptable, but I've mainly read articles and posts that say the closer the RMSE is to 0, the better, which definitely is not the case here.

Any advice is appreciated!

  • 2
    $\begingroup$ RMSE is a relative measure. There is no "good", or "bad" RMSE, no "small" or "big" RMSE, it depends on your data. You use it to compare outcomes of different models estimated on the same data, where you choose which model is "better" by choosing the one with smaller RMSE. $\endgroup$ – Tim Aug 2 '17 at 13:43

Browse other questions tagged or ask your own question.