# Feature subsampling with gradient boosting

A key component in building random forest models is feature subsampling, i.e., building each individual tree with only a percentage of predictors chosen randomly by tree. The literature often suggests a “rule of thumb” that, if there are $p$ predictors, we use $\sim\sqrt{p}$ predictors per tree in classification models or $\sim p/3$ predictors per tree in regression models (see Hastie and Tibshirani's Elements of Statistical Learning, section 15.3, for example).

My question is this: if we build a gradient boosting model that includes feature subsampling (using a package such as XGBoost), should we still adhere to this “rule of thumb” of $\sqrt{p}$ or $p/3$? In other words, in a model that “learns” as it builds trees, is it still appropriate to restrict the number of predictors per tree to the same extent as in a random forest?

Subsampling by rows and columns still increases variance between trees and allows your model to converge faster with boosting, but it is not essential. $p/3$ or $\sqrt p$ seems like it would be too low for most boosting problems. Interacted signal will be harder to find if a pair of variables have such a small chance of existing together in the same tree. I have seen occasional and marginal gains in predictive power at around $3/4$.