There is no "gradient" in the standard Random Forest formulation, but can I combine random Forests with an optimisation method like Gradient Descent or SGD?
Can I use Adam (Adaptive moment estimation) for a Random Forest?
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this communityThere is no "gradient" in the standard Random Forest formulation, but can I combine random Forests with an optimisation method like Gradient Descent or SGD?
Can I use Adam (Adaptive moment estimation) for a Random Forest?
In a sense, yes, you can.
As I alluded in my question here What *is* an Artificial Neural Network?, a decision tree is a neural network, lato sensu. So you can train these neural networks in parallel and combine then, with end-to-end gradient based optimization.
Another way you could combine random forests is through gradient boosting. Usually, for gradient boosting, "weak" learners are combined into a "strong" learner. In random forests and bagging techniques in general, "strong", biased, learners, are combined into a, hopefully, more parsimonious model. But nothing precludes you from combining random forests in gradient boosting fashion, which answers "can I combine random Forests with an optimisation method like Gradient Descent or SGD?".