I am training a
RandomForestRegressor Model with Scikit-Learn to model a physical process. The dataset has the following properties:
- 450’000 samples, 42 features
- Train test split of 80/20
When calculating the test score, I get a very high test R2 value of 0.985 and a test RMSE of 0.71. I did visualize the results in the plot below (y_pred vs. y_test):
[UPDATE2] added histogram with residuals
I am a bit suspicious, about the high test R2 value. Even though the plot indicates a relatively good fit, the test and prediction value don’t match “perfectly”. I don’t have much intuition about what to expect from the R2 value in this case.
Does anyone have an idea, if such a high R2 is plausible? Are there any sanity checks (next to using a test set), that I could apply to verify my results? Thank you!