2
$\begingroup$

So if I have have variables x1, x2, x3, x4 and I want to predict x1 using x2, x3, x4 as predictors how would the following situations affect the accuracy and overall working of the random forest algo?

  • If x2, x3, x4 aren't independent of each other
  • If x2, x3, x4 are independent but dependent on x1.
$\endgroup$
  • $\begingroup$ Regression, classification, or both? $\endgroup$ – Sean Easter Dec 15 '15 at 21:24
2
$\begingroup$

Well to obtain more non-redundant information is always better than equivalent redundant information. That said, the bootstrapping of samples for each tree and random variable subset in each node allow RF to handle collinearity fairly well.

If using bagged trees (mtry/features.tried = n.features) one variable (e.g. x2) may be used in most splits, because it was just slightly more (cor)related to x1. Thereby small non-redundant components of remaining variables (x3 and x4) may be overlooked by the model. Lowering mtry force the RF model to use all variables more equal. For mtry=1 there will roughly(not entirely accurate) be equally many splits by each variable.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.