Questions tagged [cart]

'Classification And Regression Trees', also sometimes called 'decision trees'. CART is a popular machine learning technique, and it forms the basis for techniques like random forests and common implementations of gradient boosting machines.

Filter by
Sorted by
Tagged with
4 votes
0 answers
43 views

Why are approaches that approximate a random forest with a single decision not more popular?

I understand that random forests yield better performance than standard decision trees, but are less interpretable, because they do not generate a single tree. In this question, several users provided ...
nka5we's user avatar
  • 41
0 votes
0 answers
19 views

Monotone constraints in decision tree regressor or random forest regression

after I've spent several weeks trying to fit a regression model to my flood damage data (x1=water height, x2=adaptation height, x3=(x1-x2), y=damage), it is now time for my very first question on ...
Sjafnargata's user avatar
0 votes
1 answer
17 views

Decision Tree Overfitting - Reference Request

Multiple papers and online texts start from the position that decision tree overfitting can be taken for granted. I am not here to dispute this. However, the claim of decision tree overfitting is ...
EB3112's user avatar
  • 244
0 votes
0 answers
18 views

What is the splitting criterion in Regression trees (DecisionTreeRegressor sklearn) in the multi output case

I am using DecisionTreeRegressor and RandomForestRegressor from sklearn in a case where i have multiple output, but i did not find a reference article for the regression case (which is used by sklearn)...
Rayane Elimam's user avatar
1 vote
2 answers
66 views

Sorting step in Decision and Regression Trees

Several papers and implementations require the tree building algoirithms specifically the split finding to have values of individual features in sorted order. For example, the XGBoost manuscript says ...
Karl Gardner's user avatar
0 votes
0 answers
16 views

Removing seasonality and trend for forecasting with tree based models

I am working on a problem where I'm using tree-based models (RFs, GBTs) for forecasting. I've read that I have to de-trend the data if I'm using a tree-based model, however, I am reading conflicting ...
harrynak's user avatar
0 votes
2 answers
41 views

Difference between max_depth and max_leaf_nodes in decision tree classifier (sklearn)

What is the difference between max_depth and max_leaf_nodes parameter in decision tree classifier. if depth is 4, then the number of leaf nodes will be 2^4 = 16. So providing max_depth = 4 or ...
amtn's user avatar
  • 1
2 votes
0 answers
38 views

Confusion about the code for choosing "stumps" in Adaboost algorithm

(I actually asked the following question on Stack Overflow recently: https://stackoverflow.com/questions/76842431/confusion-about-the-code-for-choosing-stumps-in-adaboost-algorithm but then I ...
Richard's user avatar
  • 121
2 votes
1 answer
123 views

Why cannot a single decision tree represent an entire Random Forest?

I was intrigued by the reply from @JohnRos to the post Making a single decision tree from a random forest. They say "<...> a random forest prediction cannot be represented by a single tree....
Smerdjakov's user avatar
1 vote
2 answers
47 views

single node tree explanation

solution: you have a single node tree and you have this explanation. "Because of the inability to split the dataset on any variable, the average of the entire data set is applied as the estimate ...
user392987's user avatar
0 votes
0 answers
25 views

2 or more continious features in Tree classification

If a training set has a continious feature, some texts recommend that first the dataset is sorted based on the continious feature, and then split points are chosen. What I am not sure about, is how ...
Karl 17302's user avatar
0 votes
0 answers
7 views

Regression models that conform to functional groupings of features

For example, suppose we want to predict y with features x1, x2, x3, x4. If I specify ...
oliffur's user avatar
  • 11
3 votes
1 answer
169 views

Can SVM and Decision Trees be seen as instances of neural networks?

We already know that neural networks with specific choices of activation function as well as connections can generalize large amount of ML models. My question is: neural network also generalize SVM ...
Fraïssé's user avatar
  • 1,373
4 votes
0 answers
15 views

Is there a decision tree impurity metric based on maximum of probabilities?

I'm trying to understand impurity metrics in decision tree learning, in particular the Gini impurity. Questioning one of the assumptions of Gini impurity has led me to another impurity measure which ...
A Kubiesa's user avatar
  • 141
0 votes
0 answers
22 views

Estimating variance of estimated mean in leaf when using Honest splitting

In Recursive Partitioning for Heterogeneous Causal Effects by Susan Athey, Guido W. Imbens, under section 2.5 Honest Splitting, two different datasets (called tr and est) are used for (a) creating the ...
fsafe's user avatar
  • 43
1 vote
0 answers
11 views

Are there any standard Goodness of fit tests for regression trees, where the output is a continuous variable?

I have fitted a regression tree on my training data and would like to demonstrate that it is a good model. For now, I am doing that by calculating the RMSE between the actual values of the dependent ...
Santanu's user avatar
  • 121
1 vote
1 answer
33 views

Goodness of fit test/index for a regression tree

I have fitted a regression tree on my data and would like to demonstrate that it is a good model. Are there any standard goodness of fit test or index for a regression tree? I understand that I can ...
Santanu's user avatar
  • 121
0 votes
1 answer
10 views

Classification criteria equation of decision trees

Can someone explain what does the term I(y=k) stand for in the equation for p_mk ?
Sherwin R's user avatar
0 votes
0 answers
25 views

Why do we really need information gain to decide the best split in decision tree?

I was thinking about the paradigm to decide the best split in decision tree, so why do we really need IG? can't we directly use the entropy of the split and decide the best split based on that? ...
Sai Sreenivas's user avatar
2 votes
1 answer
2k views

How to use categorical features in lightGBM? [closed]

I am working on an attrition dataset which has a large number of categorical parameters. Each categorical parameter has a high cardinality, so one-hot encoding them is out of question. I was looking ...
Ashish Samant's user avatar
1 vote
1 answer
53 views

Making a decision tree with numeric data

Working with decision tree and I have couple of questions: Should I always do random forest before or I can just do the decision tree, skip the random forest part? Should I always have a training ...
Sisi's user avatar
  • 61
2 votes
0 answers
35 views

Can you use regression trees for classification tasks in random forest?

I've been playing around with random forest algorithm to classify a binary Y vector using classification and regression trees. Classification trees output class probability and regression trees an ...
Mirko Pavicic's user avatar
2 votes
0 answers
36 views

Exactly when can I model a two-dimensional region with a 100% accuracy decision tree?

This is a theoritical question about decision trees. I believe that the question is very explicit in the title of the question. My thoughts on this question follow below. If the region we are dealing ...
xyz's user avatar
  • 155
2 votes
1 answer
23 views

How to find AUC from Binary Classification Decision Tree?

Decision Tree I have found Misclassification rates for all the leaf nodes. samples = 3635 + 1101 = 4736, class = Cash, misclassification rate = 1101 / 4736 = 0.232. samples = 47436 + 44556 = 91992, ...
Aman Rangapur's user avatar
0 votes
1 answer
58 views

XGBoost: Why is the "approximate algorithm" faster?

I am reading T. Chen, C. Guestrin, "XGBoost: A Scalable Tree Boosting System", 2016 (arXiv), which is seemingly full of typos. They propose the so-called "approximate algorithm" (...
paperskilltrees's user avatar
0 votes
1 answer
20 views

Multivariate analysis for subjective decision making

I am trying to find the best US state using 25 columns of normalized data (best = 1, worst = 0) such as crime rate, GDP, house prices, and others. This results in a 50x25 Excel table. Afterward, each ...
user avatar
0 votes
0 answers
22 views

Decision tree and how to make a split [duplicate]

I read that you have to follow this: maximizing the difference between the nodes and minimizing the difference within the nodes I know that SSE_parent should be bigger than SSE_child because you want ...
shawn's user avatar
  • 11
0 votes
0 answers
18 views

Decision tree for splitting a node [duplicate]

I read that you have to follow this: maximizing the difference between the nodes and minimizing the difference within the nodes I know that SSE_parent should be bigger than SSE_child because you want ...
shawn's user avatar
  • 11
2 votes
0 answers
36 views

How to identify differences between two decision trees

I have two decision trees that gives each individual a rating based on the same 4 parameters (two quantitative and one qualitative). They are supposed to be exactly the same, but give different ...
Dimitri's user avatar
  • 41
0 votes
0 answers
15 views

rpart anova continuous prediction

I have generated a continuous decision tree using rpart. The plot is: Using predict(rpart_r,newdata) I can predict the dependant variable and for one observation: ...
mpcengineer's user avatar
1 vote
0 answers
144 views

Can XGBoost handle a custom objective where the 2nd derivative can be negative?

I am going through the introduction to XGBoost page, and there is a section where they derive the optimal value of the leaf node, for a given tree structure. To quote the specific section, In this ...
Bridgeburners's user avatar
1 vote
1 answer
66 views

Rendering the decision tree as a step function [closed]

I am trying to fit a decision tree on a data with only one explanatory variable and both explanatory and response variables are continuous. I believe in such case the result tree is almost like a step ...
ok12345's user avatar
  • 13
0 votes
0 answers
14 views

What's the behaviour of decision tree classification if the Y (i.e true labels) have the same values

I know that input data is not of much use, but I still want to understand how are we going to split the data (in the decision tree) in this case (because all the features should end-up giving the same ...
Sai Sreenivas's user avatar
0 votes
0 answers
19 views

Why is the accuracy of a Decision Tree decreasing whereas the accuracy of an LSTM is increasing when adding augmented data?

I am using sklearn's DecisionTreeClassifier and LSTMs (Keras) for time series classification. To increase the accuracy and robustness of the models I augmented the ...
Unistack's user avatar
3 votes
1 answer
71 views

Does boosting being component-wise imply variable selection?

I have come across the term "component-wise" in the literature, and I am curious if this means that a model does perform variable selection. And not having this, mean it doesn't perform ...
paulmuaddib's user avatar
0 votes
0 answers
14 views

Decision trees: Measure of split quality which takes into account rare values

I am working on a classification problem in which the positive class is very rare. The dataset consists of categorical variables, as shown in the example below. The variables are hierarchical, in the ...
bjarkemoensted's user avatar
0 votes
0 answers
40 views

What is the relationship between the cp tuning parameter and the CP column in rpart output?

I am really struggling to understand the relationship between the cp tuning parameter in calls to rpart::rpart() I have read ...
Jon's user avatar
  • 159
0 votes
0 answers
30 views

Underlying decision tree in imputation(MiCE)

I am using the mice package to impute a single(factorial) varaible in a data set. Since the variable is factorial i'm using the cart method. My question is it possible to see the underlying decision ...
frikki94's user avatar
0 votes
2 answers
163 views

Combining logistic regression and decision tree?

I'm working on a project classifying patients as having (1) or not having (0) a particular condition. Someone I work with has suggested fitting a decision tree on this data, and using the leaf node ...
k13's user avatar
  • 47
1 vote
2 answers
146 views

Do random forests use weak learners (like XGBoost) or fully grown trees?

So it sounds like boosting techniques (eg. XGBoost) uses weak learners (stumps) to gradually learn sequentially. This is not in dispute I hope. However, with bagging techniques (eg. Random Forest) I'm ...
Katsu's user avatar
  • 704
0 votes
0 answers
32 views

How to model categorical variables with word frequency vectors in a decision tree?

I have a dataset that describes car failures and the action made by the mechanic to fix them. It is composed by 5 columns: Fault Code, depending on car model and car year, categorical variable that ...
Andrea Ciufo's user avatar
1 vote
1 answer
186 views

Can XGBoost do classification based on linear combinations?

Suppose we have a data set $\mathcal{D}$ consisting of $n_C$ continuous features $\boldsymbol{X}_1, \boldsymbol{X}_2, \dots, \boldsymbol{X}_{n_C}$ and we wish to target a discrete variable $\...
HeyCool08's user avatar
2 votes
1 answer
66 views

Why doesn't boosting assign higher weight to the "good" (low residual) models?

Extremely confused about the following: Lets say we start out with a dumb weak learner. Since its the 0th model and hasnt learned anything yet, we have a high residual, lets say of 10,000. We produce ...
Katsu's user avatar
  • 704
3 votes
1 answer
35 views

Determining feature importance in decision tree?

Generally speaking, if a feature is split early on in a tree does that mean its more important? And so therefore, the root node's feature is the most important feature in the tree? In addition, if a ...
Katsu's user avatar
  • 704
0 votes
0 answers
23 views

General Steps for Building a Regression Tree?

Wanted to make sure that I have the proper steps for building a Regression Decision Tree correct. Please let me know if I'm missing anything or if something looks incorrect: Build a fully grown tree: ...
Katsu's user avatar
  • 704
0 votes
0 answers
22 views

Why is variable importance for boosted trees a squared value?

I am currently trying to understand the tree variable importance calculation as proposed by Breiman; I am coming from this question and will be using its links: Relative variable importance for ...
Paul1911's user avatar
0 votes
0 answers
49 views

XGBoost Original Paper Notation for Loss Function

I'm working through the original XGBoost paper by Chen & Guestrin (2016) and I noticed they dropped a subscript i for y-hat between the first loss function and the second order approximation ...
ticiarai's user avatar
4 votes
1 answer
281 views

Why exactly do some Decision Tree Algorithms sort the features before finding the best split?

I read about the time complexity of Decision Tree Algorithms like CART, and understand why the time complexity, with sorting, can be approximated as $O(m n^2 \log n)$. I will try to go through the ...
kklaw's user avatar
  • 505
0 votes
0 answers
23 views

What can cause prediction delay in decision tree-based regressions?

I am experiencing a delay between the test data and predictions after some time: The first row is earlier than the second row. Data are residuals calculated on time series velocity data. Why does ...
Mr. Panda's user avatar
  • 263
1 vote
1 answer
90 views

Random Forest Classifier Terminal Nodes as a Probability Distribution

I've been diving deep into random forests and had a question about terminal nodes. I know in general when you reach the terminal node, or leaf, of a random forest, the assigned value for that leaf is ...
Al.Sal's user avatar
  • 113

1
2 3 4 5
26