# Is it possible to build a more "controllable" decision tree like below?

I have 2 real and 1 discrete input variable whereas the output variable takes either of the 2 nominal values (i.e. 2 class problem). First I used Weka to train C 4.5 decision tree in a 10-fold cross validation setup. As a result I get a single decision tree whose True positive, False positive rates don't change much with the pruning factor.

What I want is a single variable whose value I can threshold (in addition to class identified by decision tree) to do classification. At the same time, I want readability of decision trees.

Is it possible to do this with regression trees ? Any starting points ?

Basically I want to know if it's possible to build a decision tree like in below image -

• I'm confused. If you have a single value (gotten somehow) then the tree will have only one branch. Also, this won't tell you much - it eliminates a lot of the benefit of trees. Or are you talking about two separate runs - one tree and one single number? Aug 30, 2012 at 10:25
• Thanks for the comment. What I want is a probability / some other measure of belongingness to a particular class at each node of the tree. Aug 30, 2012 at 10:51
• @PeterFlom I have added an image to question for clarification. Aug 30, 2012 at 11:09
• Any tree procedure will give you the probability of being in a class at each node. If you want a tree method that is based on significance and such, you might look at party in R. Aug 30, 2012 at 11:09
• Thanks @Peter, I will look at the mentioned pointer. I think (correct me if I'm wrong) the probability in Weka is computed as fraction of instances classified correctly during N-fold cross validation when decision tree was being trained. If there are some other methods, which depend not merely on # instances, but are a function of input variables, it'd be more insightful. Aug 30, 2012 at 11:15

Now, what to use? I'm not exactly sure what you mean by "$x$ is a numerical attribute which corresponds to a probability of belonging to a particular class at a particular node", but if you want to regress given relative frequencies or estimated probabilities in the nodes, you can use a MOB with a beta regression model in the nodes, if you have a binary/k-ary outcome to be regressed you can use either LMT (I would not recommend it due to experience), LOTUS or MOB with binary/multinomial logistic regression.