If I have two classes that are cleanly separated by a single diagonal line as shown below, can I have a depth 1 decision tree using both features on the question x_2 > x_1 or can we not use two features in a single decision query?
$\begingroup$
$\endgroup$
0
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
I couldn't comment due to the reputation restriction. I think there are some trees built the way you question. Also, for the data you posted, we can do a rotation of data and use the regular CART.
This post explains the detail and related papers: Non-perpendicular hyperplane decision trees
$\begingroup$
$\endgroup$
V. Nice question. Since the accepted answer does not directly address the question, I am adding a few points:
- With univariate DTs, we can only segregate classes using straight lines. We could therefore build a series of vertical and horizontal lines zig zagging across the entire graph above which successfully bifurcates the two classes. Of course the depth would not be 1
- If rotation is allowed, we could rotate all data-points such that a simple straight line can now bisect them and act as the decision boundary as lovemath has suggested
- If rotation is not allowed, but the DT is multi-variate, we could in one straight diagonal line segregate the two classes. See for e.g. how https://web.njit.edu/~usman/courses/cs675_fall15/Comparing_Univariate_and_Multivariate_De.pdf seems to have used the weighted avg of the two vectors to draw a decision boundary. To envisage this, we will have to let go off the current vision we have of the traditional DTs where there are decisions taken based on comparison of attributes with certain fixed values and instead imagine using linear combination of weighted vectors to draw lines as we please at each node
