If we have numeric variable, decision trees will use < and > comparisons as splitting criteria. Lets consider this case : If our target variable is 1 for even numeric value, and 0 for odd numeric value. How to deal with this type of variables? How to even identify these type of variables if we have large number of variables? Is there any specific names for these type of variables?
$\begingroup$
$\endgroup$
5
-
$\begingroup$ the splitting occurs on the features, not the target variable. If the features are numeric, then the decision tree will conduct splitting on them with < and > in an attempt to build a model that can predict the two-class target variable. If the target is two-class, then the problem is a classification task, and the variable is called a binary variable. or are you wondering what happens in terms of splitting when one of the features is also a binary variable? $\endgroup$– develaristApr 3, 2020 at 14:43
-
$\begingroup$ Given a large enough training data set the tree will learn to distinguish between even and odd numbers. Only on numbers that occur frequently enough in the training dataset. What exactly is the question? $\endgroup$– BernhardApr 3, 2020 at 14:45
-
$\begingroup$ No, I' talking about binary classification only. With a numeric variable(predictor) values from 1 to 100. As I said our target variable is 1 for even numeric(num%2=0) value, and 0 for odd numeric value. I think in this case the tree size increase even though there is a chance to simply classify as single node with derived variable.(the derived variable is 1 if num%2=0 else 0). In this case only we obtain the classification with single node. $\endgroup$– Venkatesh GandiApr 3, 2020 at 14:48
-
$\begingroup$ I'm voting to close this question as off-topic because it is cross-posted on Data Science where it already has an answer. $\endgroup$– gung - Reinstate MonicaApr 4, 2020 at 15:41
-
$\begingroup$ cross-post link: datascience.stackexchange.com/q/71678/55122 $\endgroup$– Ben ReinigerApr 4, 2020 at 18:39
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
Given a large enough training data set the tree will learn to distinguish between even and odd numbers. Only on numbers that occur frequently enough in the training dataset.
An example in R because a plot will explain best:
# example data
x <- sample(1:6, size = 1000, replace = TRUE)
y <- ifelse(x%%2, 1, 0)
xmpl <- data.frame(x = x, y = y)
# regression
library(rpart)
model <- rpart(y ~ x, data=xmpl)
plot(model)
text(model)
If the input values range from 1 to 100 instead of 1 to 6 you will need more training data and a larger tree will grow. I am not aware of a parting algorithm that will make branches depending on whether a predictor value is even or odd nor have I ever encountered a real world problem where that would likely have solved any problem. You could probably construct such a parting algorithm if you had to.
-
$\begingroup$ Yes, That is what I'm saying the tree size will increase drastically. I hope this is not right way to deal with this variable. As I mentioned above in the comments. We can simply classify it with single node (with derived variable). Now, My question is wow to even identify these type of variables if we have large number of variables? $\endgroup$ Apr 3, 2020 at 14:59
-
$\begingroup$ I think there should be a hack to deal with these type of variables, especially for tree based algos. $\endgroup$ Apr 3, 2020 at 15:05
-
$\begingroup$ "That is what I'm saying the tree size will increase drastically." Did you say that? $\endgroup$– BernhardApr 4, 2020 at 17:53