I have implemented a random forest classifier. At the moment, I am thinking about how to tune the hyperparameters of the random forest. Of course, I am doing a gridsearch type of algorithm while checking CV errors. The problem is that I have no clue what range of the hyperparameters is even reasonable.
It is a binary classification problem, the dataset has 50 000 observations and 40 features.
I've read a paper that the RF algorithm is not able to overfit with respect to the number of trees. The more trees the better for the generalizability. Therefore, I thought I pick the number of trees fix and won't fine tune them. However, I am not sure what a reasonable number of trees would be. I guess I should consider the training data size and the number of features, but, I am still lost. Is 1000 enough?
For the subset selection of features at each node I think that I will try all values between 1 and 40. Would somebody suggest something different? I have seen that some people are just using the square root of the number of all features. However, I imagine that this might be problematic in some cases and that the correlation of the trees is not reduced sufficiently.
I control the individual tree depth by just setting a minimum number of observations in the leafs. Here I am completely lost, any suggestions? Maybe a series like 1, 5, 10, 20, 50, 100, 500, 1000, 2000, 4000, 8000, 16000, 32000? Would you control for the individual tree depth with another hyperparameter as well? Max tree levels?
Any suggestions would be helpful. I appreciate every opinion!:)