1
$\begingroup$

I am trying to run several decision tree models (CHAID, C&RT, QUEST), but I have learned that several researchers have applied logistic regression model first in order to select risk factors. So, once they have significantly factors they used them to perform a decision tree.

It is the best way to do that? Do I really need to do this before a decision tree? If so, what are the advantages?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Do you have any references? It sounds a bit odd to use logistic regression to select features for decision tree. $\endgroup$ – Zhubarb Mar 4 '15 at 8:54
  • $\begingroup$ See whether the material on CHAID at stats.stackexchange.com/questions/7815/… is helpful. $\endgroup$ – rolando2 Mar 4 '15 at 12:48
  • $\begingroup$ Zhurbarb, this is the reference that talks about doing Logistic regression before decision tree model: biomedcentral.com/1472-6963/14/382 $\endgroup$ – Rogelio Pujol Mar 5 '15 at 11:35
1
$\begingroup$

Doing feature selection based on statistical significance is a bad idea.

There are no real advantages to doing this. With enough data, all effects will be significant, so you would end up selecting all the variables. Not only that, but the p values doesn't tell you anything about the size of the effect so you might end up selecting features which effect the outcome in a negligible way.

There is no need to do this.

Improve this answer
$\endgroup$
0
$\begingroup$

Perhaps some are thinking it as a way to do variable selection via automated stepwise or other methods, but I think it might hide some problems related to data minining results quality.

Why not do first independently all models and do some quality checking via ROC-curves or such?

It could also be proper to do ensembling, such as averaging probability estimates from the various models.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I'd be interested in an expanded version of this answer. $\endgroup$ – rolando2 Mar 4 '15 at 12:51
  • $\begingroup$ "It could also be proper to do ensembling, such as averaging probability estimates from the various models." -- this is effectively what a random forest does. $\endgroup$ – Victor Stoddard Jul 4 '17 at 22:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.