3
$\begingroup$

What I mean is, constructing a decision tree where each node is not a single feature instead composition of multiple features, therefore the evaluation criteria should consider a metric over multiple numbers instead of single number. For the original case of decision tree we are looking for a feature's value comparing its ordinal relation < or > to other instances. But I am expecting to compare multiple value like $x_1,\cdots,x_n$ with new instance's values $x_1,\cdots,x_n$ at each node selection. It is like to have an vector of values selected for each decision-tree node.

$\endgroup$
1
$\begingroup$

The general strategy to accomplish this within the context of decision trees is to create dummy variables representing the various forms of variable interaction. However, you can achieve the same result (mostly) by the various techniques in random forest implementations.

If you're asking whether a decision tree can create new data to decide on based on the prior node's result, you will have to write your own algorithm to add a transformation step prior to each split.

$\endgroup$
0
$\begingroup$

There are several attempts to incorporate alternative variations of splitting rules into decisions trees via generalizing the CART algorithm. One prominent one is GUIDE by Wei-Yin Loh in Wisconsin, another is the rotation forest which has also been extended to RandomForest ensembles. My experience with rotation forest hasn't been great byt your mileage may vary. I also haven't used the GUIDE algorithm either, as noted in the 1988 JASA paper on GUIDE the computational complexity is increased by considering linear combinations of features. Also with rotation forest the same complaint with PCA also holds which is that you lose the interpretability of the splits when considering linear combinations that satisfy an orthogonality constraint.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.