Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 0 down vote favorite

Running a logistic regression we get p-values for all the input variables which helps us choose significant inputs. Similarly can we use the classification trees to pick variables that are split, and use those variables in the model? I think the fact that splitting the dataset on a variable leads to lower classification error should be a good indicator of the predictive power of the variable, is that true?

share|improve this question
1  
Some smart-aleck is going to answer "Yes" (which, incidentally, I believe is the right answer). Is there more that you are really intending to ask? – rolando2 Apr 24 '12 at 2:09

1 Answer

up vote 3 down vote

It is generally not valid to "choose significant inputs", a form of highly problematic stepwise regression. Using classification trees to form inputs for logistic regression is even more problematic because of simultaneously increasing both type I and type II error.

share|improve this answer
+1 Maybe it would help the OP if you could explain why and how "choosing significant inputs " leads to invalid conclusions and how type I and type II risk get increased, for what test and how "simultaneously" so. – Momo Jun 23 '12 at 17:20
also +1, @Momo, I agree w/ your suggestions here. For the interim, I provided a conceptual overview of why using 'significance' leads to problems here. – gung Jun 23 '12 at 17:32
2  
Choosing "significant inputs" is double dipping. It systematically biases the model in favor of extreme predictions and destroys type I error. Tree methods increase type II error because of their ineffective way of handling continuous predictors and additivity. The link from @gung is excellent. To address the greatly increased type I error you would need very complex adjustment, but this wouldn't help with the type II error or with the generally poor future predictive performance of such approaches. – Frank Harrell Jun 23 '12 at 17:54
@FrankHarrell, thanks for the complement, that's very kind of you. I didn't know that about CART, but it makes sense; I finally purchased a copy of your book (I got tired of the fact that it's always checked out at the library) so maybe I'll understand these things better soon. – gung Jun 23 '12 at 18:15
Thanks @frankharrel from me too. – Momo Jun 23 '12 at 18:39

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.