I did gridsearch with corss-validation on a trainingset to search for best hyperparameters for a Random Forest Regressor. And indeed the best parameterset gives good results in cross-validation (R^2 ~ 86%, which is slightly better than my previous results). The results on my Testset are quite similar, so everything seems fine to me.
BUT: The Hyperparameter for the minimum of samples per leaf is set to 1, which often leads to massive overfitting. And indeed, when applying the model to the trainingset, R^2 is 99%.
Now I wonder wether this is a problem? Everywhere you read that you should avoid overfitting. And when I trained a neural net on the same dataset, I had to regularize by dropout, so that R^2 on Testset (e.g. 80%) and on Trainingset (e.g. 90%) could meet in the midlle at around 85%. But when regularizing the random forest, R^2 on Trainingset AND Testset decreases.
So is overfitting here a problem, when the Random Forest Regressor (that definitly overfits with 99% to 86%) does a good job in cross-validation and Testset anyways? Or what would you do?
Thank you for your answers!