Questions tagged [cart]
'Classification And Regression Trees', also sometimes called 'decision trees'. CART is a popular machine learning technique, and it forms the basis for techniques like random forests and common implementations of gradient boosting machines.
1,261
questions
4
votes
0
answers
43
views
Why are approaches that approximate a random forest with a single decision not more popular?
I understand that random forests yield better performance than standard decision trees, but are less interpretable, because they do not generate a single tree. In this question, several users provided ...
- 41
0
votes
0
answers
19
views
Monotone constraints in decision tree regressor or random forest regression
after I've spent several weeks trying to fit a regression model to my flood damage data (x1=water height, x2=adaptation height, x3=(x1-x2), y=damage), it is now time for my very first question on ...
- 11
0
votes
1
answer
17
views
Decision Tree Overfitting - Reference Request
Multiple papers and online texts start from the position that decision tree overfitting can be taken for granted.
I am not here to dispute this. However, the claim of decision tree overfitting is ...
- 244
0
votes
0
answers
18
views
What is the splitting criterion in Regression trees (DecisionTreeRegressor sklearn) in the multi output case
I am using DecisionTreeRegressor and RandomForestRegressor from sklearn in a case where i have multiple output, but i did not find a reference article for the regression case (which is used by sklearn)...
1
vote
2
answers
66
views
Sorting step in Decision and Regression Trees
Several papers and implementations require the tree building algoirithms specifically the split finding to have values of individual features in sorted order. For example, the XGBoost manuscript says ...
- 11
0
votes
0
answers
16
views
Removing seasonality and trend for forecasting with tree based models
I am working on a problem where I'm using tree-based models (RFs, GBTs) for forecasting. I've read that I have to de-trend the data if I'm using a tree-based model, however, I am reading conflicting ...
- 3
0
votes
2
answers
41
views
Difference between max_depth and max_leaf_nodes in decision tree classifier (sklearn)
What is the difference between max_depth and max_leaf_nodes parameter in decision tree classifier.
if depth is 4, then the number of leaf nodes will be 2^4 = 16.
So providing max_depth = 4 or ...
- 1
2
votes
0
answers
38
views
Confusion about the code for choosing "stumps" in Adaboost algorithm
(I actually asked the following question on Stack Overflow recently:
https://stackoverflow.com/questions/76842431/confusion-about-the-code-for-choosing-stumps-in-adaboost-algorithm
but then I ...
- 121
2
votes
1
answer
123
views
Why cannot a single decision tree represent an entire Random Forest?
I was intrigued by the reply from @JohnRos to the post Making a single decision tree from a random forest.
They say "<...> a random forest prediction cannot be represented by a single tree....
- 73
1
vote
2
answers
47
views
single node tree explanation
solution: you have a single node tree and you have this explanation. "Because of the inability to split the dataset on any variable, the average of the entire data set is applied as the estimate ...
- 11
0
votes
0
answers
25
views
2 or more continious features in Tree classification
If a training set has a continious feature, some texts recommend that first the dataset is sorted based on the continious feature, and then split points are chosen. What I am not sure about, is how ...
0
votes
0
answers
7
views
Regression models that conform to functional groupings of features
For example, suppose we want to predict y with features x1, x2, x3, x4. If I specify
...
- 11
3
votes
1
answer
169
views
Can SVM and Decision Trees be seen as instances of neural networks?
We already know that neural networks with specific choices of activation function as well as connections can generalize large amount of ML models.
My question is: neural network also generalize SVM ...
- 1,373
4
votes
0
answers
15
views
Is there a decision tree impurity metric based on maximum of probabilities?
I'm trying to understand impurity metrics in decision tree learning, in particular the Gini impurity. Questioning one of the assumptions of Gini impurity has led me to another impurity measure which ...
- 141
0
votes
0
answers
22
views
Estimating variance of estimated mean in leaf when using Honest splitting
In Recursive Partitioning for Heterogeneous Causal Effects by Susan Athey, Guido W. Imbens, under section 2.5 Honest Splitting, two different datasets (called tr and est) are used for (a) creating the ...
- 43
1
vote
0
answers
11
views
Are there any standard Goodness of fit tests for regression trees, where the output is a continuous variable?
I have fitted a regression tree on my training data and would like to demonstrate that it is a good model. For now, I am doing that by calculating the RMSE between the actual values of the dependent ...
- 121
1
vote
1
answer
33
views
Goodness of fit test/index for a regression tree
I have fitted a regression tree on my data and would like to demonstrate that it is a good model. Are there any standard goodness of fit test or index for a regression tree?
I understand that I can ...
- 121
0
votes
1
answer
10
views
Classification criteria equation of decision trees
Can someone explain what does the term I(y=k) stand for in the equation for p_mk ?
- 11
0
votes
0
answers
25
views
Why do we really need information gain to decide the best split in decision tree?
I was thinking about the paradigm to decide the best split in decision tree, so why do we really need IG? can't we directly use the entropy of the split and decide the best split based on that?
...
- 131
2
votes
1
answer
2k
views
How to use categorical features in lightGBM? [closed]
I am working on an attrition dataset which has a large number of categorical parameters. Each categorical parameter has a high cardinality, so one-hot encoding them is out of question. I was looking ...
1
vote
1
answer
53
views
Making a decision tree with numeric data
Working with decision tree and I have couple of questions:
Should I always do random forest before or I can just do the decision tree, skip the random forest part?
Should I always have a training ...
- 61
2
votes
0
answers
35
views
Can you use regression trees for classification tasks in random forest?
I've been playing around with random forest algorithm to classify a binary Y vector using classification and regression trees. Classification trees output class probability and regression trees an ...
2
votes
0
answers
36
views
Exactly when can I model a two-dimensional region with a 100% accuracy decision tree?
This is a theoritical question about decision trees. I believe that the question is very explicit in the title of the question. My thoughts on this question follow below.
If the region we are dealing ...
- 155
2
votes
1
answer
23
views
How to find AUC from Binary Classification Decision Tree?
Decision Tree
I have found Misclassification rates for all the leaf nodes.
samples = 3635 + 1101 = 4736, class = Cash, misclassification rate = 1101 / 4736 = 0.232.
samples = 47436 + 44556 = 91992, ...
0
votes
1
answer
58
views
XGBoost: Why is the "approximate algorithm" faster?
I am reading T. Chen, C. Guestrin, "XGBoost: A Scalable Tree Boosting System", 2016 (arXiv), which is seemingly full of typos. They propose the so-called "approximate algorithm" (...
- 315
0
votes
1
answer
20
views
Multivariate analysis for subjective decision making
I am trying to find the best US state using 25 columns of normalized data (best = 1, worst = 0) such as crime rate, GDP, house prices, and others. This results in a 50x25 Excel table. Afterward, each ...
Neil Peña Martínez
0
votes
0
answers
22
views
Decision tree and how to make a split [duplicate]
I read that you have to follow this: maximizing the difference between the nodes and minimizing the difference within the nodes
I know that SSE_parent should be bigger than SSE_child because you want ...
- 11
0
votes
0
answers
18
views
Decision tree for splitting a node [duplicate]
I read that you have to follow this: maximizing the difference between the nodes and minimizing the difference within the nodes
I know that SSE_parent should be bigger than SSE_child because you want ...
- 11
2
votes
0
answers
36
views
How to identify differences between two decision trees
I have two decision trees that gives each individual a rating based on the same 4 parameters (two quantitative and one qualitative). They are supposed to be exactly the same, but give different ...
- 41
0
votes
0
answers
15
views
rpart anova continuous prediction
I have generated a continuous decision tree using rpart.
The plot is:
Using predict(rpart_r,newdata) I can predict the dependant variable and for one observation:
...
1
vote
0
answers
144
views
Can XGBoost handle a custom objective where the 2nd derivative can be negative?
I am going through the introduction to XGBoost page, and there is a section where they derive the optimal value of the leaf node, for a given tree structure.
To quote the specific section,
In this ...
- 3,374
1
vote
1
answer
66
views
Rendering the decision tree as a step function [closed]
I am trying to fit a decision tree on a data with only one explanatory variable and both explanatory and response variables are continuous. I believe in such case the result tree is almost like a step ...
- 13
0
votes
0
answers
14
views
What's the behaviour of decision tree classification if the Y (i.e true labels) have the same values
I know that input data is not of much use, but I still want to understand how are we going to split the data (in the decision tree) in this case (because all the features should end-up giving the same ...
- 131
0
votes
0
answers
19
views
Why is the accuracy of a Decision Tree decreasing whereas the accuracy of an LSTM is increasing when adding augmented data?
I am using sklearn's DecisionTreeClassifier and LSTMs (Keras) for time series classification. To increase the accuracy and robustness of the models I augmented the ...
- 31
3
votes
1
answer
71
views
Does boosting being component-wise imply variable selection?
I have come across the term "component-wise" in the literature, and I am curious if this means that a model does perform variable selection. And not having this, mean it doesn't perform ...
- 33
0
votes
0
answers
14
views
Decision trees: Measure of split quality which takes into account rare values
I am working on a classification problem in which the positive class is very rare.
The dataset consists of categorical variables, as shown in the example below.
The variables are hierarchical, in the ...
- 462
0
votes
0
answers
40
views
What is the relationship between the cp tuning parameter and the CP column in rpart output?
I am really struggling to understand the relationship between the cp tuning parameter in calls to rpart::rpart() I have read ...
- 159
0
votes
0
answers
30
views
Underlying decision tree in imputation(MiCE)
I am using the mice package to impute a single(factorial) varaible in a data set. Since the variable is factorial i'm using the cart method. My question is it possible to see the underlying decision ...
- 1
0
votes
2
answers
163
views
Combining logistic regression and decision tree?
I'm working on a project classifying patients as having (1) or not having (0) a particular condition. Someone I work with has suggested fitting a decision tree on this data, and using the leaf node ...
- 47
1
vote
2
answers
146
views
Do random forests use weak learners (like XGBoost) or fully grown trees?
So it sounds like boosting techniques (eg. XGBoost) uses weak learners (stumps) to gradually learn sequentially. This is not in dispute I hope.
However, with bagging techniques (eg. Random Forest) I'm ...
- 704
0
votes
0
answers
32
views
How to model categorical variables with word frequency vectors in a decision tree?
I have a dataset that describes car failures and the action made by the mechanic to fix them.
It is composed by 5 columns:
Fault Code, depending on car model and car year, categorical variable that ...
- 111
1
vote
1
answer
186
views
Can XGBoost do classification based on linear combinations?
Suppose we have a data set $\mathcal{D}$ consisting of $n_C$ continuous features $\boldsymbol{X}_1, \boldsymbol{X}_2, \dots, \boldsymbol{X}_{n_C}$ and we wish to target a discrete variable $\...
- 25
2
votes
1
answer
66
views
Why doesn't boosting assign higher weight to the "good" (low residual) models?
Extremely confused about the following:
Lets say we start out with a dumb weak learner. Since its the 0th model and hasnt learned anything yet, we have a high residual, lets say of 10,000.
We produce ...
- 704
3
votes
1
answer
35
views
Determining feature importance in decision tree?
Generally speaking, if a feature is split early on in a tree does that mean its more important? And so therefore, the root node's feature is the most important feature in the tree?
In addition, if a ...
- 704
0
votes
0
answers
23
views
General Steps for Building a Regression Tree?
Wanted to make sure that I have the proper steps for building a Regression Decision Tree correct. Please let me know if I'm missing anything or if something looks incorrect:
Build a fully grown tree:
...
- 704
0
votes
0
answers
22
views
Why is variable importance for boosted trees a squared value?
I am currently trying to understand the tree variable importance calculation as proposed by Breiman; I am coming from this question and will be using its links: Relative variable importance for ...
- 13
0
votes
0
answers
49
views
XGBoost Original Paper Notation for Loss Function
I'm working through the original XGBoost paper by Chen & Guestrin (2016) and I noticed they dropped a subscript i for y-hat between the first loss function and the second order approximation ...
- 1
4
votes
1
answer
281
views
Why exactly do some Decision Tree Algorithms sort the features before finding the best split?
I read about the time complexity of Decision Tree Algorithms like CART, and understand why the time complexity, with sorting, can be approximated as $O(m n^2 \log n)$.
I will try to go through the ...
- 505
0
votes
0
answers
23
views
What can cause prediction delay in decision tree-based regressions?
I am experiencing a delay between the test data and predictions after some time:
The first row is earlier than the second row. Data are residuals calculated on time series velocity data.
Why does ...
- 263
1
vote
1
answer
90
views
Random Forest Classifier Terminal Nodes as a Probability Distribution
I've been diving deep into random forests and had a question about terminal nodes.
I know in general when you reach the terminal node, or leaf, of a random forest, the assigned value for that leaf is ...
- 113