# How to interpret when logistic regression is performing better than classification trees?

I used both techniques on a rather unbalance data set (90%/10%) with 2500 observations and about 20 features. Firstly, I compared the default models. In a second step, I weighed the misclassification costs differently to correct for the skewed distribution. However, classification trees kept to perform worse. Before weighing the costs differently, the trees kept splitting only once (after pruning). How should I conclude here? A few points which crossed my mind are the following:

• the problem might be kind of linearly separable and this is why logistic regression is performing better
• the features do not interact with each other
• the features can simply not explain the response (but this brings up the question of why the variables are significant in the logistic model)
• "90/10" means that you have only 100 observations? Then a tree model will be very shallow, while a logistic regression can retain more variables before getting into overestimation. So I am not very surprised that logistic regression is better! – kjetil b halvorsen Jan 7 '16 at 0:06
• Don't be mislead by classification error, which is an improper accuracy scoring rule. But you would expect logistic regression to almost always beat single trees. That's why random forests were invented. – Frank Harrell Jan 7 '16 at 3:36
• Thank you for the fast replies! However, I nean 90%/10%! I'm sorry for being not so clear. – Patrick Balada Jan 7 '16 at 8:00

I would guess that you happen have a problem where the $f(x)=Pr[Y=1\ \|\ X=x]$ is very smooth in x, and furthermore approximately obeys the logistic curve. The fact that the trees only split once seems to support this; there aren't sharp jumps and dips in $f(x)$ to make cuts on.