1
$\begingroup$

So I'm applying Random forest regression from sklearn library to a dataset having only one feature and I'm getting a very good score. The output labels are continuous.

The problem is I don't quite understand how are the decision trees being built here. While building a decision tree we split at each node based on a feature. since we have only feature here, will the decision trees have only one node? How will the decision trees be different?

$\endgroup$
1
$\begingroup$

The trees will have several nodes as the real line is recursively split. Your model is free to make multiple splits on the same variable. Depending on the parameters of the estimator, your estimator will:

  1. Bootstrap a new dataset from your training data
  2. Randomly determine the splitting criterion based on a random sample of features. Since you only have a single feature, your model will always split based on that feature. Thus, your model is really just a bag of trees.
| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

Random Forests use multiple kinds of randomization.

  • In each split, they only consider a small subset of possible features. If you have only one feature, then this will of course always be used. Randomization in this sense thus doesn't have an impact, as you write.

  • However, Random Forests also bootstrap: each decision tree is grown on a bootstrapped resample of the original data. This kind of randomization of course applies to your situation.

    The advantage of the bootstrap is that each tree will split on the predictor at different values. The Random Forest can therefore model nonlinearities in the feature-response relationship (assuming enough data).

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.