I am an absolute beginner in field of machine learning, I started doing titanic assignment in Kaggle and found(read some where) Random Forest is the best fit. I started reading about random forest and found the Explanation by Edwin Chen in this question intuitive. This made me "understand" how I can solve the Titanic assignment which predicts if one survives or not(classification). But I cannot understand How random Forest will work for regression which is continuous.

Please don't mind to point out any mistakes in my assumptions or the way I started things. Any advice would be helpful, This looks very vast and Don't even know where to begin.


Basically there are two differences:

  • when building the model/tree it is used a different criteria to split information; for example on binary split the purpose is to choose the split variable and split value so that the sum of variances of the two resulting data points on target/output variable is minimal
  • when you predict values you will use the mean value of the target/output variable for all data points in the leaf node

Some variants I saw:

  • for splitting you might want to minimize the sum of standard deviations, the weighed sum of variances, etc
  • for prediction values you can also use trimmed mean, median or even another model (like a linear model fitted on instances from the node)

Ignore the forest part for a moment, even a single tree can do regression. Each leaf holds a prediction value, which no longer is a class for regression. Given an input feature vector, you simply walk the tree as you'd do for a classification problem, and the resulting value in the leaf node is the prediction.

For a forest, simply averaging the prediction of each tree is valid, although you may want to investigate if that's sufficiently robust for your application.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.