Questions tagged [cart]

'Classification And Regression Trees'. CART is a popular machine learning technique, and it forms the basis for techniques like random forests and common implementations of gradient boosting machines.

Filter by
Sorted by
Tagged with
0 votes
0 answers
4 views

How does sklearn.tree.DecisionTreeRegressor work?

I have successfully trained the model on a dataset, but I have some questions because the documentation here is very difficult to read: Does the splitter use a single scalar among the inputs, or ...
user avatar
  • 111
0 votes
0 answers
20 views

Decision tree feature importances on a test set

Many tree based models come with a built in feature importance method usually based on impurity decrease (ex. sklearn RandomForest). Would it be possible to calculate feature importances on a test set ...
user avatar
  • 1
2 votes
1 answer
30 views

XGBoost interpretation of the plot in R

I am applying XGBoost implementation in R on the data with 9 columns. After training the model, I tried to plot the "multiple-in-one" tree using the xgb.plot.multi.trees() function with the ...
user avatar
  • 21
0 votes
0 answers
9 views

As alpha increases with a tree model, what happens to its flexibility? [closed]

Suppose there is a tree model with $\alpha = 0$ and another with $\alpha = 1$. What happens to the flexibility as the value increases from $0$ to $1$? I suspect that it will have lower flexibility and ...
user avatar
0 votes
0 answers
7 views

What is the appropriate type for the variables to be included in CART models?

I am building a CART model. However, I have certain doubts about the type of the explanatory variables to be included in the model. I ask you for your help and advice on these two specific questions: ...
user avatar
0 votes
0 answers
3 views

How to prepare constructs for inputting in a deicion tree from likert scale items

I have to use 5 variables in my decision tree as predictor variables. Each variable is an average score of Likert scale items as follows: Self-control is a construct of 10 variables. Each of these 10 ...
user avatar
0 votes
0 answers
27 views

Rules, theoreticial basis on selecting which machine learning model acceptable to be combined into a voting classifier

Background: Voting ensembles (hard/maximum voting, averaging/soft voting) and stack models are considered as the ensemble technique that can improve individual performance of the machine learning ...
user avatar
1 vote
0 answers
22 views

What is the reason for using the mean instead a regression inside a decision tree?

A common problem with decisions trees is that they don't generalize outside the range of the data, e.g. time series. For this reason, I want to code a regression version of the decision tree, that ...
user avatar
  • 131
0 votes
0 answers
22 views

How to fix the tree structure for a tree-based algorithm?

Background Some of our BI analysts and most of our managers are interested in making explainable predictions. One of our colleagues proposed an approach based on individual tree leaves from a tree-...
user avatar
  • 153
0 votes
0 answers
13 views

Counting number of possible decision trees

I spent a lot of time on this question, and I search extensively on the internet, but I cannot find an answer (only an unanswered question here). Assume we have $p$ binary covariates, so that for each ...
user avatar
2 votes
1 answer
32 views

Adj. $R^2$ with tree ensembles

Consider tree ensemble methods such as XGBoost, Lightgbm and/or Catboost. Is the adj. $R^2$ a valid metric for tree ensembles? I'm curious because these methods handle factor variables differently. E....
user avatar
  • 218
0 votes
0 answers
24 views

Computational Complexity of Gradient Boosting Decision Tree algorithm

Does anybody have an idea about the computational complexity of GBDT? I have only seen one paper report (Gradient Boosted Decision Trees for High Dimensional Sparse Output). It doesn't seem to be ...
user avatar
1 vote
1 answer
17 views

What to do after pruning the decision tree?

Suppose we stop growing the tree here at gender Then if a new sample has moderate education will consider the label to be true or false? Looks like it will be false 66% of the time but this isn't ...
user avatar
  • 167
2 votes
1 answer
45 views

How to interpret output of rpart decision tree?

I need your help please in interpreting this: I am trying to predict suicide rate but confused about interpretation. for number 2 for example, for the group age 5-14, the suicide rate will be 0.35 ...
user avatar
0 votes
0 answers
16 views

Why does the slow version of TreeExplainer algorithm have a complexity of $TLM2^M$

I'm trying to understand the TreeExplainer algorithms better from From Local Explanations to Global Understanding with Explainable AI for Trees, but confused by how the time complexity is derived even ...
user avatar
  • 887
0 votes
0 answers
12 views

Alternative to post or pre-pruning

Usually, to better generalize and have a better understandability of the underlying model, we prune the decision trees. And in some cases, it is still large and difficult to interpret. Is there any ...
user avatar
  • 279
0 votes
0 answers
36 views

CART coverall accuracy vs. RF & SVM

I am performing a supervised classification with RF, SVM, and CART algorithms. I have over 2000 training points in an area of 9,995 km². For CART, I have obtained a 'Validation overall accuracy = 1' ...
user avatar
1 vote
1 answer
17 views

Permutation test for mob() tree in partykit

I have data of $N\approx 1200$ whereby treated and control individuals have been matched (via full-matching) as a pre-processing step. This matching step induces correlations between treated and ...
user avatar
4 votes
2 answers
230 views

Can decision stumps have more than 2 leaves?

I understand decision stump: a shallow 1-level decision tree is often used as base-leaner in ensemble methods such as AdaBoost. What is not immediately clear to me ...
user avatar
  • 231
1 vote
1 answer
40 views

Why will the estimates of prediction error typically be biased upward with Cross-Validation?

Why the estimates of prediction error will typically be biased upward with Cross-Validation? Is it like with decisions tree? Using a stopping criterion will increase a little the bias but will ...
user avatar
1 vote
1 answer
32 views

How is overfitting reduced when more examples are added?

I've developed a model and it's overfitting. I want to understand whether adding more instances to my data will reduce overfitting by making the model weaker on the training data or by improving the ...
user avatar
6 votes
3 answers
440 views

XGBoost Classifier not capturing extreme probabilities

I'm using XGBoost for a binary classification task—trying to predict whether team A will beat team B given the score of the game and the time left. I know for certain score-time combinations, the ...
user avatar
0 votes
0 answers
19 views

Kernelized Decision Trees

I came across a simple example that shows where decision trees may have difficulty solving a classification problem efficiently: "[...] For example, if we have a two-class problem and the ...
user avatar
0 votes
0 answers
12 views

Can I use the learning set as a test set for bootstrapped predictors?

This question relates to Leo Breiman's paper: Bagging Predictors from 1996. The author claims that if bagging is deployed, the original training set can be used to assess the performance of the ...
user avatar
1 vote
1 answer
20 views

How does the performance of bagging depend on instability?

This question relates to Leo Breiman's paper: Bagging Predictors from 1996. Assuming that $\mathcal{L}$ denotes the training set and $\phi$ the predictor which depends on the training set and the ...
user avatar
1 vote
0 answers
55 views

How do non binary decision trees deal with categorical values that weren't in training?

I've been implementing Random Forest from scratch as a learning exercise. While most algorithms for decision trees seem to deal exclusively in binary yes/no questions, leading to binary trees, ...
user avatar
  • 11
0 votes
0 answers
9 views

Predicitions from conditional inference forests for ordinal responses

How do I get class predictions from a conditional inference forest (utilizing ordinal regression trees) for an ordinal response? By majority vote or average or to classify into the most likely class? ...
user avatar
  • 1
0 votes
0 answers
11 views

Estimating heterogeneous causal effects using causal trees

Can someone here explain the difference between transformed outcome trees (TOT), causal trees, and fit-based trees? I want to use trees for heterogeneous treatment effects and I am unsure as to which ...
user avatar
1 vote
1 answer
38 views

can tree-based models extrapolate with categorical independent variables

I am aware of the fact that tree-based (machine learning) models struggle to extrapolate - see regression example here. I am only familiar with the concept of extrapolation for numeric independent ...
user avatar
  • 1,326
3 votes
2 answers
421 views

Why use supervised binning on train data if it leaks data?

I have a dataset which has Quantity ordered (along with other variables like product type, product price, customer group etc). Target variable is whether customer ...
user avatar
  • 1,804
0 votes
1 answer
33 views

Can we Categorize Different Statistical and Machine Learning Models as "P vs NP"? [closed]

In the context of Computer Science and Optimization, I have heard that different problems can be classified using the "P vs NP" framework. Essentially, there is a hierarchy of problems based ...
user avatar
  • 5,688
0 votes
0 answers
21 views

Poisson tree fit an intercept only

Looking at this answer by Achim Zeileis: ...
user avatar
  • 1,326
1 vote
1 answer
38 views

What's the difference between a recommender system and a decision tree?

We learned in Machine Learning that both of those techniques try to predict an output (whether person A likes a specific product or whether person A has a high default risk) based on data of other ...
user avatar
0 votes
0 answers
7 views

Is there an efficient way of finding the best feature to split on before calculating the losses of all features? (Decision trees)

When building a decision tree with multiple real-valued features, how do you decide on which feature to split on? Is there a fast way to determine? One way I can think of is calculating the best cut-...
user avatar
0 votes
0 answers
19 views

Comparison between regression models based on trees

I'm solving a prediction problem in which I have an independent variables Y and 13 dependent variables which also are highly correlated. My dataset is composed by 124 observation for the train dataset ...
user avatar
  • 1
0 votes
0 answers
47 views

How to select the best variables to train the best model for regression problems? [duplicate]

How to choose the best variables? I'm training a regression model with single tree and ensemble methods (bagging and random forest) to make a prediction. In the exploration phase I found different ...
user avatar
0 votes
1 answer
36 views

Which kind of distributions can decision trees learn (well)?

Suppose I have a classification task in $\mathbb{R}^d$, given by a distribution $P$ and training data $D$. The decision boundary is $$B = \{x \; : \; P(Y=1|x) = P(Y=0|x)\}$$ Assume I am learning a ...
user avatar
0 votes
0 answers
7 views

Could decision trees be used for time series, modeling change in input? [duplicate]

I'm curious if decision trees and variants (random forest, boosted trees, etc.) could be used for time series tasks. I know that these methods tend to predict the mean observations in a given leaf and ...
user avatar
  • 1,700
0 votes
0 answers
21 views

Plotting variable importance for CART in R using bar graphs

How can I plot variable importance for a decision tree (CART) in R? Since I am new to R, I need the code (if possible, I want to plot the relative importance score for each variable using bar graphs). ...
user avatar
0 votes
0 answers
11 views

Existing library to build decision tree on sparse data

My apologies if this is too much of a repeat of another question I asked, but the question is important, and I am trying to learn from my experience of asking this question to ask it the right way. In ...
user avatar
  • 101
0 votes
0 answers
48 views

Why does a Decision Tree algorithm outperform the Random Forest Algorithm in certain cases?

Currently I write my master thesis that deals with the binary prediction of university dropouts (dropout - yes/no). In the thesis, I compare the performance of three different classification ...
user avatar
0 votes
0 answers
49 views

Best practice for cyclical feature encoding for tree-based methods

Right now I'm facing a dataset of professional road cyclists which contains training data of the athletes at different dates over two years. For some tree-based methods I'm still looking for a 'best ...
user avatar
  • 218
2 votes
1 answer
21 views

Influence function used in partykit for binary classification

What is the influence function used for binary classification in the R package partkit, specifically for the conditional tree (...
user avatar
  • 1,045
0 votes
0 answers
11 views

What statistic/algorithm does the decision tree use to partition/cluster the data per depth?

How does a decision tree calculate that a break exists at 1.8 on the root, a break exists at 2.1 and 1.2 on the second depth? I know Gini and Entropy are used to calculate which feature to partition ...
user avatar
  • 224
0 votes
0 answers
35 views

How are surrogate splits used by rpart for variable importance when there are no missing data?

I have read the rpart vignette/ longintro. Its not clear how variable importance is measured especially this line: An overall measure of variable importance is the sum of the goodness of split ...
user avatar
  • 1
0 votes
0 answers
21 views

Constrained Clustering via Distance-Based Multivariate Regression Trees in R?

I was wondering if distance-based multivariate regression trees (or distance-based multivariate random forests) are implemented any R package? De'ath (2002) describes multivariate regression trees ...
user avatar
  • 33
0 votes
0 answers
13 views

Optimize parametric Log-Likelihood with a Decision Tree

Suppose there are some objects with features, and the target is parametric density estimation. Density estimation is model-based. Parameters are obtained by maximizing log-likelihood. $LL = \sum_{i \...
user avatar
1 vote
0 answers
19 views

Why it's slow to build tree for forward stagewise additive model?

According to Elementary Statistical Learning https://hastie.su.domains/Papers/ESLII.pdf Page 357. The author pointed there exists no simple method to do tree induction for loss function like Huber ...
user avatar
  • 43
4 votes
3 answers
437 views

Does it make sense to have 2 branches resulting in same decision while using one single variable to divide the predictor multiple times?

I have been recently come across a problem that entails using a decision tree that only uses one continuous variable to divide the predictor on multiple threshold, while some splits result in the same ...
user avatar
1 vote
1 answer
36 views

Small dataset results in regression tree with only 1 terminal node

I don't understand why this decision tree has only one terminal node: ...
user avatar

1
2 3 4 5
25