Questions tagged [cart]
'Classification And Regression Trees'. CART is a popular machine learning technique, and it forms the basis for techniques like random forests and common implementations of gradient boosting machines.
1,202
questions
0
votes
0
answers
4
views
How does sklearn.tree.DecisionTreeRegressor work?
I have successfully trained the model on a dataset, but I have some questions because the documentation here is very difficult to read:
Does the splitter use a single scalar among the inputs, or ...
- 111
0
votes
0
answers
20
views
Decision tree feature importances on a test set
Many tree based models come with a built in feature importance method usually based on impurity decrease (ex. sklearn RandomForest). Would it be possible to calculate feature importances on a test set ...
- 1
2
votes
1
answer
30
views
XGBoost interpretation of the plot in R
I am applying XGBoost implementation in R on the data with 9 columns. After training the model, I tried to plot the "multiple-in-one" tree using the xgb.plot.multi.trees() function with the ...
- 21
0
votes
0
answers
9
views
As alpha increases with a tree model, what happens to its flexibility? [closed]
Suppose there is a tree model with $\alpha = 0$ and another with $\alpha = 1$.
What happens to the flexibility as the value increases from $0$ to $1$?
I suspect that it will have lower flexibility and ...
0
votes
0
answers
7
views
What is the appropriate type for the variables to be included in CART models?
I am building a CART model. However, I have certain doubts about the type of the explanatory variables to be included in the model. I ask you for your help and advice on these two specific questions:
...
- 1
0
votes
0
answers
3
views
How to prepare constructs for inputting in a deicion tree from likert scale items
I have to use 5 variables in my decision tree as predictor variables. Each variable is an average score of Likert scale items as follows:
Self-control is a construct of 10 variables. Each of these 10 ...
- 119
0
votes
0
answers
27
views
Rules, theoreticial basis on selecting which machine learning model acceptable to be combined into a voting classifier
Background:
Voting ensembles (hard/maximum voting, averaging/soft voting) and stack models are considered as the ensemble technique that can improve individual performance of the machine learning ...
1
vote
0
answers
22
views
What is the reason for using the mean instead a regression inside a decision tree?
A common problem with decisions trees is that they don't generalize outside the range of the data, e.g. time series. For this reason, I want to code a regression version of the decision tree, that ...
- 131
0
votes
0
answers
22
views
How to fix the tree structure for a tree-based algorithm?
Background
Some of our BI analysts and most of our managers are interested in making explainable predictions. One of our colleagues proposed an approach based on individual tree leaves from a tree-...
- 153
0
votes
0
answers
13
views
Counting number of possible decision trees
I spent a lot of time on this question, and I search extensively on the internet, but I cannot find an answer (only an unanswered question here).
Assume we have $p$ binary covariates, so that for each ...
- 352
2
votes
1
answer
32
views
Adj. $R^2$ with tree ensembles
Consider tree ensemble methods such as XGBoost, Lightgbm and/or Catboost.
Is the adj. $R^2$ a valid metric for tree ensembles?
I'm curious because these methods handle factor variables differently. E....
- 218
0
votes
0
answers
24
views
Computational Complexity of Gradient Boosting Decision Tree algorithm
Does anybody have an idea about the computational complexity of GBDT? I have only seen one paper report (Gradient Boosted Decision Trees for High Dimensional Sparse Output). It doesn't seem to be ...
1
vote
1
answer
17
views
What to do after pruning the decision tree?
Suppose we stop growing the tree here at gender
Then if a new sample has moderate education will consider the label to be true or false? Looks like it will be false 66% of the time but this isn't ...
- 167
2
votes
1
answer
45
views
How to interpret output of rpart decision tree?
I need your help please in interpreting this: I am trying to predict suicide rate but confused about interpretation. for number 2 for example, for the group age 5-14, the suicide rate will be 0.35 ...
Souad
0
votes
0
answers
16
views
Why does the slow version of TreeExplainer algorithm have a complexity of $TLM2^M$
I'm trying to understand the TreeExplainer algorithms better from From Local Explanations to Global Understanding with Explainable AI for Trees, but confused by how the time complexity is derived even ...
- 887
0
votes
0
answers
12
views
Alternative to post or pre-pruning
Usually, to better generalize and have a better understandability of the underlying model, we prune the decision trees. And in some cases, it is still large and difficult to interpret. Is there any ...
- 279
0
votes
0
answers
36
views
CART coverall accuracy vs. RF & SVM
I am performing a supervised classification with RF, SVM, and CART algorithms.
I have over 2000 training points in an area of 9,995 km².
For CART, I have obtained a 'Validation overall accuracy = 1' ...
Paris
1
vote
1
answer
17
views
Permutation test for mob() tree in partykit
I have data of $N\approx 1200$ whereby treated and control individuals have been matched (via full-matching) as a pre-processing step. This matching step induces correlations between treated and ...
- 663
4
votes
2
answers
230
views
Can decision stumps have more than 2 leaves?
I understand decision stump: a shallow 1-level decision tree is often used as base-leaner in ensemble methods such as AdaBoost. What is not immediately clear to me ...
- 231
1
vote
1
answer
40
views
Why will the estimates of prediction error typically be biased upward with Cross-Validation?
Why the estimates of prediction error will typically be biased upward with Cross-Validation?
Is it like with decisions tree? Using a stopping criterion will increase a little the bias but will ...
- 33
1
vote
1
answer
32
views
How is overfitting reduced when more examples are added?
I've developed a model and it's overfitting. I want to understand whether adding more instances to my data will reduce overfitting by making the model weaker on the training data or by improving the ...
- 137
6
votes
3
answers
440
views
XGBoost Classifier not capturing extreme probabilities
I'm using XGBoost for a binary classification task—trying to predict whether team A will beat team B given the score of the game and the time left. I know for certain score-time combinations, the ...
0
votes
0
answers
19
views
Kernelized Decision Trees
I came across a simple example that shows where decision trees may have difficulty solving a classification problem efficiently:
"[...] For example, if we have a two-class problem and the ...
- 21
0
votes
0
answers
12
views
Can I use the learning set as a test set for bootstrapped predictors?
This question relates to Leo Breiman's paper: Bagging Predictors from 1996.
The author claims that if bagging is deployed, the original training set can be used to assess the performance of the ...
- 21
1
vote
1
answer
20
views
How does the performance of bagging depend on instability?
This question relates to Leo Breiman's paper: Bagging Predictors from 1996.
Assuming that $\mathcal{L}$ denotes the training set and $\phi$ the predictor which depends on the training set and the ...
- 21
1
vote
0
answers
55
views
How do non binary decision trees deal with categorical values that weren't in training?
I've been implementing Random Forest from scratch as a learning exercise.
While most algorithms for decision trees seem to deal exclusively in binary yes/no questions, leading to binary trees, ...
- 11
0
votes
0
answers
9
views
Predicitions from conditional inference forests for ordinal responses
How do I get class predictions from a conditional inference forest (utilizing ordinal regression trees) for an ordinal response? By majority vote or average or to classify into the most likely class?
...
- 1
0
votes
0
answers
11
views
Estimating heterogeneous causal effects using causal trees
Can someone here explain the difference between transformed outcome trees (TOT), causal trees, and fit-based trees? I want to use trees for heterogeneous treatment effects and I am unsure as to which ...
- 119
1
vote
1
answer
38
views
can tree-based models extrapolate with categorical independent variables
I am aware of the fact that tree-based (machine learning) models struggle to extrapolate - see regression example here. I am only familiar with the concept of extrapolation for numeric independent ...
- 1,326
3
votes
2
answers
421
views
Why use supervised binning on train data if it leaks data?
I have a dataset which has Quantity ordered (along with other variables like product type, product price, customer group etc). Target variable is whether customer ...
- 1,804
0
votes
1
answer
33
views
Can we Categorize Different Statistical and Machine Learning Models as "P vs NP"? [closed]
In the context of Computer Science and Optimization, I have heard that different problems can be classified using the "P vs NP" framework. Essentially, there is a hierarchy of problems based ...
- 5,688
0
votes
0
answers
21
views
1
vote
1
answer
38
views
What's the difference between a recommender system and a decision tree?
We learned in Machine Learning that both of those techniques try to predict an output (whether person A likes a specific product or whether person A has a high default risk) based on data of other ...
0
votes
0
answers
7
views
Is there an efficient way of finding the best feature to split on before calculating the losses of all features? (Decision trees)
When building a decision tree with multiple real-valued features, how do you decide on which feature to split on? Is there a fast way to determine? One way I can think of is calculating the best cut-...
- 203
0
votes
0
answers
19
views
Comparison between regression models based on trees
I'm solving a prediction problem in which I have an independent variables Y and 13 dependent variables which also are highly correlated.
My dataset is composed by 124 observation for the train dataset ...
- 1
0
votes
0
answers
47
views
How to select the best variables to train the best model for regression problems? [duplicate]
How to choose the best variables?
I'm training a regression model with single tree and ensemble methods (bagging and random forest) to make a prediction. In the exploration phase I found different ...
casta
0
votes
1
answer
36
views
Which kind of distributions can decision trees learn (well)?
Suppose I have a classification task in $\mathbb{R}^d$, given by a distribution $P$ and training data $D$. The decision boundary is
$$B = \{x \; : \; P(Y=1|x) = P(Y=0|x)\}$$
Assume I am learning a ...
- 229
0
votes
0
answers
7
views
Could decision trees be used for time series, modeling change in input? [duplicate]
I'm curious if decision trees and variants (random forest, boosted trees, etc.) could be used for time series tasks. I know that these methods tend to predict the mean observations in a given leaf and ...
- 1,700
0
votes
0
answers
21
views
Plotting variable importance for CART in R using bar graphs
How can I plot variable importance for a decision tree (CART) in R? Since I am new to R, I need the code (if possible, I want to plot the relative importance score for each variable using bar graphs). ...
- 119
0
votes
0
answers
11
views
Existing library to build decision tree on sparse data
My apologies if this is too much of a repeat of another question I asked, but the question is important, and I am trying to learn from my experience of asking this question to ask it the right way.
In ...
- 101
0
votes
0
answers
48
views
Why does a Decision Tree algorithm outperform the Random Forest Algorithm in certain cases?
Currently I write my master thesis that deals with the binary prediction of university dropouts (dropout - yes/no). In the thesis, I compare the performance of three different classification ...
- 11
0
votes
0
answers
49
views
Best practice for cyclical feature encoding for tree-based methods
Right now I'm facing a dataset of professional road cyclists which contains training data of the athletes at different dates over two years. For some tree-based methods I'm still looking for a 'best ...
- 218
2
votes
1
answer
21
views
Influence function used in partykit for binary classification
What is the influence function used for binary classification in the R package partkit, specifically for the conditional tree (...
- 1,045
0
votes
0
answers
11
views
What statistic/algorithm does the decision tree use to partition/cluster the data per depth?
How does a decision tree calculate that a break exists at 1.8 on the root, a break exists at 2.1 and 1.2 on the second depth?
I know Gini and Entropy are used to calculate which feature to partition ...
- 224
0
votes
0
answers
35
views
How are surrogate splits used by rpart for variable importance when there are no missing data?
I have read the rpart vignette/ longintro. Its not clear how variable importance is measured especially this line:
An overall measure of variable importance is the sum of the goodness of split
...
- 1
0
votes
0
answers
21
views
Constrained Clustering via Distance-Based Multivariate Regression Trees in R?
I was wondering if distance-based multivariate regression trees (or distance-based multivariate random forests) are implemented any R package? De'ath (2002) describes multivariate regression trees ...
- 33
0
votes
0
answers
13
views
Optimize parametric Log-Likelihood with a Decision Tree
Suppose there are some objects with features, and the target is parametric density estimation. Density estimation is model-based. Parameters are obtained by maximizing log-likelihood.
$LL = \sum_{i \...
- 1
1
vote
0
answers
19
views
Why it's slow to build tree for forward stagewise additive model?
According to Elementary Statistical Learning https://hastie.su.domains/Papers/ESLII.pdf Page 357.
The author pointed there exists no simple method to do tree induction for loss function like Huber ...
- 43
4
votes
3
answers
437
views
Does it make sense to have 2 branches resulting in same decision while using one single variable to divide the predictor multiple times?
I have been recently come across a problem that entails using a decision tree that only uses one continuous variable to divide the predictor on multiple threshold, while some splits result in the same ...
- 233
1
vote
1
answer
36
views
Small dataset results in regression tree with only 1 terminal node
I don't understand why this decision tree has only one terminal node:
...
- 1,372