Questions tagged [cart]

'Classification And Regression Trees', also sometimes called 'decision trees'. CART is a popular machine learning technique, and it forms the basis for techniques like random forests and common implementations of gradient boosting machines.

Filter by
Sorted by
Tagged with
1 vote
0 answers
26 views

XGB predict_proba estimates don't match sum of leaves [closed]

When using an XGB model in the context of binary classification, I observed that the test estimates given by predict_proba were close but not equal to the results I ...
Juan Felipe Salamanca Lozano's user avatar
0 votes
1 answer
19 views

Standardise features for a tree based model or logistic regression

I am trying to understand if one should standardise features for all models and when does it make sense to do so. Is the below statement true? If yes, could you give a bit of explanation please. ...
learner's user avatar
  • 617
0 votes
0 answers
17 views

How to calculate the statistic for ctree function?

...
Apai's user avatar
  • 1
-1 votes
0 answers
25 views

How to use uneven communication for binary prediction

I have a sensor that captures when there was an impact in a container. With this i have a dataset that contains data of when the device communicated and there was no impact and when the device ...
Wwwardrunaa's user avatar
0 votes
1 answer
16 views

Understanding rpart Tree Splits: Why Did a Node Remain Terminal Despite Potential Split?

Setup Consider two trees grown with rpart() from the rpart R package, in which the only thing that changes is the minbucket ...
Ewen Gallic's user avatar
0 votes
0 answers
14 views

Missing features in decision tree based algorithms

I have a medium-sized dataset consisting of many features, some of which can contain missing values. I want to predict a variable using an algorithm that employs decision trees (specifically XGBoost, ...
umbal's user avatar
  • 65
2 votes
1 answer
236 views

Can you explain this description of tree pruning in Intro to Statistical Learning?

The underlined sentences below from p. 331 in An Introduction to Statistical Learning have me scratching my head: Given that the splitting algorithm always finds the best next split in terms of error ...
Jon's user avatar
  • 181
0 votes
0 answers
19 views

Dataset for impact prediction in a container

I have a sensor that captures when there was an impact in a container. With this i have a dataset that contains data of when the device communicated and there was no impact and when the device ...
Wwwardrunaa's user avatar
0 votes
0 answers
9 views

What are the problem with running binary classification on panel data using logistic regression?

I am a newbie and I am trying to understand why one cannot run algorithms like logistic regression or decision trees on panel dataset ? I am running an employee attrition model where the dependent ...
learner's user avatar
  • 617
3 votes
1 answer
41 views

Features available during training but not at prediction

Broadly, my motivation is to understand if/how features available during training but not at prediction can be used to improve the prediction accuracy of a machine learning model. This question is ...
Foster's user avatar
  • 31
0 votes
1 answer
35 views

How does Cross Validation work in decision trees (or tree ensembles)

I've been working with tree-based models for a long time and I never really asked myself how cross-validation would work when building a tree. For the sake of this question, suppose I've split my ...
Arturo Sbr's user avatar
0 votes
1 answer
32 views

How to handle correlated variables before using Recursive Feature Elimination?

I have seen a few Kaggle notebooks that list without reason that RFE works better when removing correlated variables. I struggle to see the reason why so I conducted some of my own research and would ...
AvanishM's user avatar
1 vote
0 answers
79 views

Why does feature importance decrease for highly correlated variables?

I am investigating the relationship between correlation between features and its impact on their feature importances using sklearn's DecisionTreeClassifier algorithm. I manipulated the correlation of ...
AvanishM's user avatar
0 votes
0 answers
18 views

Decision Tree: What do you do if all your classes have same frequency or you have no data points at the leaf node?

I was solving a question and I realized after splitting the tree on every attribute. I reached a point where if my final attribute value is "No" then all the classes have equal frequency. if ...
Archaic's user avatar
0 votes
0 answers
11 views

Cubist Model Tree Overlapping Rules

Can someone help me understand the output of a cubist model created by the R Cubist package? The documentation package manual for objects of class "cubist" seems to be nonexistent. Looking ...
noNameTed's user avatar
  • 113
0 votes
0 answers
39 views

Fitting probabilities for decision trees

I perform an accident analysis with decision trees. Some cases are clear (all info exists), other ones haven’t it. By my previous experience, I can add missing data (restoration costs). But I don’t ...
manro's user avatar
  • 101
0 votes
0 answers
13 views

Ensemble, merge or combine multiple lmertree objects

Working with the PISA data, which includes multiple achievement scores (plausible values) for each participant, I would like to run the same lmertree and ensemble ...
Burak Aydin's user avatar
2 votes
2 answers
72 views

Why does my neural network consider different features important compared to my decision tree?

I built a neural network and a decision tree using very similar data sets (the only difference was the randomness of selecting the training vs testing set). The variables with the highest shapley ...
Jay's user avatar
  • 41
1 vote
0 answers
38 views

How does random forest determine the possible split values to assess?

I understand the basics of how decision trees / random forests are built, but one thing I still don't have a good answer for is how the putative split values are identified. So suppose we have mtry = ...
user3037237's user avatar
0 votes
2 answers
63 views

Classification and regression trees splitting depth - how it works?

I am trying to understand how a CART tree grows, So I am growing a tree step by step, and I am finding a strange (?) behavior. Let me show this by means of an example: I will use the titanic data set ...
Nicolas Molano's user avatar
0 votes
0 answers
33 views

Derivation of the formula for E(MSE) in Recursive Partitioning for Heterogeneous Causal Effects (Athey and Imben 2015)

This post answers a question from the Recursive Partitioning for Heterogeneous Causal Effects paper by Athey and Imbens. However, I am blocked at an even earlier stage from the previous post. I don't ...
timothee_stat's user avatar
0 votes
0 answers
33 views

How to use a random forest on a continuous dependent variable (sales) to determine absolute contribution of each variable to performance?

The ultimate question I'm trying to answer is as follows: Given year-on-year sales change (i.e. 2023 sales minus 2022 sales), split out by a number of variables, how can I calculate the absolute ...
Sam Eley's user avatar
2 votes
1 answer
136 views

Gini impurity greedily optimises a loss function in decision trees

I am trying to understand how the Gini criterion for decision decision tree construction actually greedily optimises a loss function. The Gini impurity, sometimes also called Gini index, for a region (...
ngmir's user avatar
  • 341
4 votes
0 answers
86 views

Why are approaches that approximate a random forest with a single decision not more popular?

I understand that random forests yield better performance than standard decision trees, but are less interpretable, because they do not generate a single tree. In this question, several users provided ...
nka5we's user avatar
  • 49
0 votes
0 answers
272 views

Monotone constraints in decision tree regressor or random forest regression

after I've spent several weeks trying to fit a regression model to my flood damage data (x1=water height, x2=adaptation height, x3=(x1-x2), y=damage), it is now time for my very first question on ...
Sjafnargata's user avatar
0 votes
1 answer
20 views

Decision Tree Overfitting - Reference Request

Multiple papers and online texts start from the position that decision tree overfitting can be taken for granted. I am not here to dispute this. However, the claim of decision tree overfitting is ...
EB3112's user avatar
  • 244
0 votes
0 answers
55 views

What is the splitting criterion in Regression trees (DecisionTreeRegressor sklearn) in the multi output case

I am using DecisionTreeRegressor and RandomForestRegressor from sklearn in a case where i have multiple output, but i did not find a reference article for the regression case (which is used by sklearn)...
Rayane Elimam's user avatar
1 vote
2 answers
93 views

Sorting step in Decision and Regression Trees

Several papers and implementations require the tree building algoirithms specifically the split finding to have values of individual features in sorted order. For example, the XGBoost manuscript says ...
Karl Gardner's user avatar
0 votes
0 answers
20 views

Removing seasonality and trend for forecasting with tree based models

I am working on a problem where I'm using tree-based models (RFs, GBTs) for forecasting. I've read that I have to de-trend the data if I'm using a tree-based model, however, I am reading conflicting ...
harrynak's user avatar
0 votes
2 answers
489 views

Difference between max_depth and max_leaf_nodes in decision tree classifier (sklearn)

What is the difference between max_depth and max_leaf_nodes parameter in decision tree classifier. if depth is 4, then the number of leaf nodes will be 2^4 = 16. So providing max_depth = 4 or ...
amtn's user avatar
  • 1
2 votes
0 answers
46 views

Confusion about the code for choosing "stumps" in Adaboost algorithm

(I actually asked the following question on Stack Overflow recently: https://stackoverflow.com/questions/76842431/confusion-about-the-code-for-choosing-stumps-in-adaboost-algorithm but then I ...
Richard's user avatar
  • 121
3 votes
1 answer
164 views

Why cannot a single decision tree represent an entire Random Forest?

I was intrigued by the reply from @JohnRos to the post Making a single decision tree from a random forest. They say "<...> a random forest prediction cannot be represented by a single tree....
Smerdjakov's user avatar
1 vote
2 answers
146 views

single node tree explanation

solution: you have a single node tree and you have this explanation. "Because of the inability to split the dataset on any variable, the average of the entire data set is applied as the estimate ...
user392987's user avatar
0 votes
0 answers
26 views

2 or more continious features in Tree classification

If a training set has a continious feature, some texts recommend that first the dataset is sorted based on the continious feature, and then split points are chosen. What I am not sure about, is how ...
Karl 17302's user avatar
0 votes
0 answers
7 views

Regression models that conform to functional groupings of features

For example, suppose we want to predict y with features x1, x2, x3, x4. If I specify ...
olives's user avatar
  • 31
3 votes
1 answer
209 views

Can SVM and Decision Trees be seen as instances of neural networks?

We already know that neural networks with specific choices of activation function as well as connections can generalize large amount of ML models. My question is: neural network also generalize SVM ...
Fraïssé's user avatar
  • 1,550
4 votes
0 answers
16 views

Is there a decision tree impurity metric based on maximum of probabilities?

I'm trying to understand impurity metrics in decision tree learning, in particular the Gini impurity. Questioning one of the assumptions of Gini impurity has led me to another impurity measure which ...
A Kubiesa's user avatar
  • 141
0 votes
0 answers
42 views

Estimating variance of estimated mean in leaf when using Honest splitting

In Recursive Partitioning for Heterogeneous Causal Effects by Susan Athey, Guido W. Imbens, under section 2.5 Honest Splitting, two different datasets (called tr and est) are used for (a) creating the ...
fsafe's user avatar
  • 43
1 vote
0 answers
13 views

Are there any standard Goodness of fit tests for regression trees, where the output is a continuous variable?

I have fitted a regression tree on my training data and would like to demonstrate that it is a good model. For now, I am doing that by calculating the RMSE between the actual values of the dependent ...
Santanu's user avatar
  • 121
1 vote
1 answer
89 views

Goodness of fit test/index for a regression tree

I have fitted a regression tree on my data and would like to demonstrate that it is a good model. Are there any standard goodness of fit test or index for a regression tree? I understand that I can ...
Santanu's user avatar
  • 121
0 votes
1 answer
15 views

Classification criteria equation of decision trees

Can someone explain what does the term I(y=k) stand for in the equation for p_mk ?
Sherwin R's user avatar
0 votes
0 answers
37 views

Why do we really need information gain to decide the best split in decision tree?

I was thinking about the paradigm to decide the best split in decision tree, so why do we really need IG? can't we directly use the entropy of the split and decide the best split based on that? ...
Sai Sreenivas's user avatar
2 votes
1 answer
6k views

How to use categorical features in lightGBM? [closed]

I am working on an attrition dataset which has a large number of categorical parameters. Each categorical parameter has a high cardinality, so one-hot encoding them is out of question. I was looking ...
Ashish Samant's user avatar
1 vote
1 answer
60 views

Making a decision tree with numeric data

Working with decision tree and I have couple of questions: Should I always do random forest before or I can just do the decision tree, skip the random forest part? Should I always have a training ...
Sisi's user avatar
  • 61
2 votes
0 answers
39 views

Can you use regression trees for classification tasks in random forest?

I've been playing around with random forest algorithm to classify a binary Y vector using classification and regression trees. Classification trees output class probability and regression trees an ...
Mirko Pavicic's user avatar
3 votes
0 answers
43 views

Exactly when can I model a two-dimensional region with a 100% accuracy decision tree?

This is a theoritical question about decision trees. I believe that the question is very explicit in the title of the question. My thoughts on this question follow below. If the region we are dealing ...
xyz's user avatar
  • 185
2 votes
1 answer
46 views

How to find AUC from Binary Classification Decision Tree?

Decision Tree I have found Misclassification rates for all the leaf nodes. samples = 3635 + 1101 = 4736, class = Cash, misclassification rate = 1101 / 4736 = 0.232. samples = 47436 + 44556 = 91992, ...
Aman Rangapur's user avatar
1 vote
1 answer
71 views

XGBoost: Why is the "approximate algorithm" faster?

I am reading T. Chen, C. Guestrin, "XGBoost: A Scalable Tree Boosting System", 2016 (arXiv), which is seemingly full of typos. They propose the so-called "approximate algorithm" (...
paperskilltrees's user avatar
0 votes
1 answer
22 views

Multivariate analysis for subjective decision making

I am trying to find the best US state using 25 columns of normalized data (best = 1, worst = 0) such as crime rate, GDP, house prices, and others. This results in a 50x25 Excel table. Afterward, each ...
user avatar
2 votes
0 answers
57 views

How to identify differences between two decision trees

I have two decision trees that gives each individual a rating based on the same 4 parameters (two quantitative and one qualitative). They are supposed to be exactly the same, but give different ...
Dimitri's user avatar
  • 41

1
2 3 4 5
26