What is the reason behind major difference in selection of mtry in RandomForest for Classification & regression? The default/recommended value of mtry is P/3 for regression task while it is SquareRoot(P) for a classification task. (where P is number of variables)
As per my understanding the fundamental idea behind RF is using smaller subset of variables in Random Forest is to create weak & diverse classifiers and aggregating them into one stronger classifier.
But I could not find any specific documentation on why a regression task has default of P/3 instead of SquareRoot(P).
Is it something specific to Random Forest?If so what is it?
OR
Is it more fundamental like difference between performance/construction of regression/classification trees which dictates RF.
I'm not looking for a complete simplified answer, just point me to right literature links :)
Thanks!
 A: The only useful source I found for this is the original paper of RF itself: http://machinelearning202.pbworks.com/w/file/fetch/60606349/breiman_randomforests.pdf
To quote "An interesting difference between regression and classification is that the
correlation increases quite slowly as the number of features used increases.
The major effect is the decrease in PE*( tree). Therefore, a relatively large
number of features are required to reduce PE*(tree) and get near optimal testset
error."
So basically in classification the strength did not increase much with increasing features for the split but the correlation did, so they recommend using less number of features. While in regression the strength of the tree increases(error decreases) while correlation increases slowly so more number of features are used for optimal performance.
I guess you could just read their experiments on different datasets with number of features for both classification and regression and draw your own conclusion.
A: Good defaults of hyperparameters in machine learning algorithms have to be found empirically on datasets (if there would be a good theory for setting them, they would not be a hyperparameter anymore). 
Probably it has shown good performances for the creator of the specific package on some datasets, so he has chosen this value. 
I am doing some studies on many datasets and one of my aims is to find the best defaults in general. 
