Skip to main content

Questions tagged [cart]

'Classification And Regression Trees', also sometimes called 'decision trees'. CART is a popular machine learning technique, and it forms the basis for techniques like random forests and common implementations of gradient boosting machines.

Filter by
Sorted by
Tagged with
2 votes
2 answers
62 views

Is feature importance given by decision tree universal?

I'm wondering that if I have a set of features on a fitted classification decision tree with relative low feature importance, would it mean that these features would also be negligible when fitted ...
bachts's user avatar
  • 21
0 votes
0 answers
10 views

What time window to use for independent variables?

I have 10000 listings on various days. My y variable is % of transactions that happened 7 days before listing with a loss > 0 is 30% or more then 1 else 0. Now losses can happen before listing also....
Mnag's user avatar
  • 1
2 votes
0 answers
346 views

How is Gini impurity related to accuracy when predicting the majority class?

For simplicity, consider the binary case, where we have a set of elements with each element belonging to one of two classes (0 or 1). Let p(j) be the proportion of ...
alex76's user avatar
  • 21
1 vote
0 answers
26 views

XGB predict_proba estimates don't match sum of leaves [closed]

When using an XGB model in the context of binary classification, I observed that the test estimates given by predict_proba were close but not equal to the results I ...
Juan Felipe Salamanca Lozano's user avatar
0 votes
1 answer
19 views

Standardise features for a tree based model or logistic regression

I am trying to understand if one should standardise features for all models and when does it make sense to do so. Is the below statement true? If yes, could you give a bit of explanation please. ...
learner's user avatar
  • 617
0 votes
0 answers
17 views

How to calculate the statistic for ctree function?

...
Apai's user avatar
  • 19
0 votes
1 answer
18 views

Understanding rpart Tree Splits: Why Did a Node Remain Terminal Despite Potential Split?

Setup Consider two trees grown with rpart() from the rpart R package, in which the only thing that changes is the minbucket ...
Ewen Gallic's user avatar
0 votes
0 answers
16 views

Missing features in decision tree based algorithms

I have a medium-sized dataset consisting of many features, some of which can contain missing values. I want to predict a variable using an algorithm that employs decision trees (specifically XGBoost, ...
umbal's user avatar
  • 65
2 votes
1 answer
238 views

Can you explain this description of tree pruning in Intro to Statistical Learning?

The underlined sentences below from p. 331 in An Introduction to Statistical Learning have me scratching my head: Given that the splitting algorithm always finds the best next split in terms of error ...
Jon's user avatar
  • 181
0 votes
0 answers
19 views

Dataset for impact prediction in a container

I have a sensor that captures when there was an impact in a container. With this i have a dataset that contains data of when the device communicated and there was no impact and when the device ...
Wwwardrunaa's user avatar
0 votes
0 answers
9 views

What are the problem with running binary classification on panel data using logistic regression?

I am a newbie and I am trying to understand why one cannot run algorithms like logistic regression or decision trees on panel dataset ? I am running an employee attrition model where the dependent ...
learner's user avatar
  • 617
3 votes
1 answer
45 views

Features available during training but not at prediction

Broadly, my motivation is to understand if/how features available during training but not at prediction can be used to improve the prediction accuracy of a machine learning model. This question is ...
Foster's user avatar
  • 31
0 votes
1 answer
42 views

How does Cross Validation work in decision trees (or tree ensembles)

I've been working with tree-based models for a long time and I never really asked myself how cross-validation would work when building a tree. For the sake of this question, suppose I've split my ...
Arturo Sbr's user avatar
0 votes
1 answer
39 views

How to handle correlated variables before using Recursive Feature Elimination?

I have seen a few Kaggle notebooks that list without reason that RFE works better when removing correlated variables. I struggle to see the reason why so I conducted some of my own research and would ...
AvanishM's user avatar
1 vote
0 answers
82 views

Why does feature importance decrease for highly correlated variables?

I am investigating the relationship between correlation between features and its impact on their feature importances using sklearn's DecisionTreeClassifier algorithm. I manipulated the correlation of ...
AvanishM's user avatar
0 votes
0 answers
20 views

Decision Tree: What do you do if all your classes have same frequency or you have no data points at the leaf node?

I was solving a question and I realized after splitting the tree on every attribute. I reached a point where if my final attribute value is "No" then all the classes have equal frequency. if ...
Archaic's user avatar
0 votes
0 answers
11 views

Cubist Model Tree Overlapping Rules

Can someone help me understand the output of a cubist model created by the R Cubist package? The documentation package manual for objects of class "cubist" seems to be nonexistent. Looking ...
noNameTed's user avatar
  • 133
0 votes
0 answers
39 views

Fitting probabilities for decision trees

I perform an accident analysis with decision trees. Some cases are clear (all info exists), other ones haven’t it. By my previous experience, I can add missing data (restoration costs). But I don’t ...
manro's user avatar
  • 101
0 votes
0 answers
13 views

Ensemble, merge or combine multiple lmertree objects

Working with the PISA data, which includes multiple achievement scores (plausible values) for each participant, I would like to run the same lmertree and ensemble ...
Burak Aydin's user avatar
2 votes
2 answers
75 views

Why does my neural network consider different features important compared to my decision tree?

I built a neural network and a decision tree using very similar data sets (the only difference was the randomness of selecting the training vs testing set). The variables with the highest shapley ...
Jay's user avatar
  • 41
1 vote
0 answers
39 views

How does random forest determine the possible split values to assess?

I understand the basics of how decision trees / random forests are built, but one thing I still don't have a good answer for is how the putative split values are identified. So suppose we have mtry = ...
user3037237's user avatar
0 votes
2 answers
65 views

Classification and regression trees splitting depth - how it works?

I am trying to understand how a CART tree grows, So I am growing a tree step by step, and I am finding a strange (?) behavior. Let me show this by means of an example: I will use the titanic data set ...
Nicolas Molano's user avatar
0 votes
0 answers
39 views

Derivation of the formula for E(MSE) in Recursive Partitioning for Heterogeneous Causal Effects (Athey and Imben 2015)

This post answers a question from the Recursive Partitioning for Heterogeneous Causal Effects paper by Athey and Imbens. However, I am blocked at an even earlier stage from the previous post. I don't ...
timothee_stat's user avatar
0 votes
0 answers
34 views

How to use a random forest on a continuous dependent variable (sales) to determine absolute contribution of each variable to performance?

The ultimate question I'm trying to answer is as follows: Given year-on-year sales change (i.e. 2023 sales minus 2022 sales), split out by a number of variables, how can I calculate the absolute ...
Sam Eley's user avatar
2 votes
1 answer
151 views

Gini impurity greedily optimises a loss function in decision trees

I am trying to understand how the Gini criterion for decision decision tree construction actually greedily optimises a loss function. The Gini impurity, sometimes also called Gini index, for a region (...
ngmir's user avatar
  • 339
4 votes
0 answers
88 views

Why are approaches that approximate a random forest with a single decision not more popular?

I understand that random forests yield better performance than standard decision trees, but are less interpretable, because they do not generate a single tree. In this question, several users provided ...
nka5we's user avatar
  • 49
0 votes
0 answers
312 views

Monotone constraints in decision tree regressor or random forest regression

after I've spent several weeks trying to fit a regression model to my flood damage data (x1=water height, x2=adaptation height, x3=(x1-x2), y=damage), it is now time for my very first question on ...
Sjafnargata's user avatar
0 votes
1 answer
21 views

Decision Tree Overfitting - Reference Request

Multiple papers and online texts start from the position that decision tree overfitting can be taken for granted. I am not here to dispute this. However, the claim of decision tree overfitting is ...
EB3112's user avatar
  • 244
0 votes
0 answers
56 views

What is the splitting criterion in Regression trees (DecisionTreeRegressor sklearn) in the multi output case

I am using DecisionTreeRegressor and RandomForestRegressor from sklearn in a case where i have multiple output, but i did not find a reference article for the regression case (which is used by sklearn)...
Rayane Elimam's user avatar
1 vote
2 answers
100 views

Sorting step in Decision and Regression Trees

Several papers and implementations require the tree building algoirithms specifically the split finding to have values of individual features in sorted order. For example, the XGBoost manuscript says ...
Karl Gardner's user avatar
0 votes
0 answers
21 views

Removing seasonality and trend for forecasting with tree based models

I am working on a problem where I'm using tree-based models (RFs, GBTs) for forecasting. I've read that I have to de-trend the data if I'm using a tree-based model, however, I am reading conflicting ...
harrynak's user avatar
0 votes
2 answers
609 views

Difference between max_depth and max_leaf_nodes in decision tree classifier (sklearn)

What is the difference between max_depth and max_leaf_nodes parameter in decision tree classifier. if depth is 4, then the number of leaf nodes will be 2^4 = 16. So providing max_depth = 4 or ...
amtn's user avatar
  • 1
2 votes
0 answers
47 views

Confusion about the code for choosing "stumps" in Adaboost algorithm

(I actually asked the following question on Stack Overflow recently: https://stackoverflow.com/questions/76842431/confusion-about-the-code-for-choosing-stumps-in-adaboost-algorithm but then I ...
Richard's user avatar
  • 121
3 votes
1 answer
170 views

Why cannot a single decision tree represent an entire Random Forest?

I was intrigued by the reply from @JohnRos to the post Making a single decision tree from a random forest. They say "<...> a random forest prediction cannot be represented by a single tree....
Smerdjakov's user avatar
1 vote
2 answers
181 views

single node tree explanation

solution: you have a single node tree and you have this explanation. "Because of the inability to split the dataset on any variable, the average of the entire data set is applied as the estimate ...
user392987's user avatar
0 votes
0 answers
26 views

2 or more continious features in Tree classification

If a training set has a continious feature, some texts recommend that first the dataset is sorted based on the continious feature, and then split points are chosen. What I am not sure about, is how ...
Karl 17302's user avatar
0 votes
0 answers
7 views

Regression models that conform to functional groupings of features

For example, suppose we want to predict y with features x1, x2, x3, x4. If I specify ...
olives's user avatar
  • 31
3 votes
1 answer
220 views

Can SVM and Decision Trees be seen as instances of neural networks?

We already know that neural networks with specific choices of activation function as well as connections can generalize large amount of ML models. My question is: neural network also generalize SVM ...
Fraïssé's user avatar
  • 1,560
4 votes
0 answers
16 views

Is there a decision tree impurity metric based on maximum of probabilities?

I'm trying to understand impurity metrics in decision tree learning, in particular the Gini impurity. Questioning one of the assumptions of Gini impurity has led me to another impurity measure which ...
A Kubiesa's user avatar
  • 141
0 votes
0 answers
43 views

Estimating variance of estimated mean in leaf when using Honest splitting

In Recursive Partitioning for Heterogeneous Causal Effects by Susan Athey, Guido W. Imbens, under section 2.5 Honest Splitting, two different datasets (called tr and est) are used for (a) creating the ...
fsafe's user avatar
  • 43
1 vote
0 answers
13 views

Are there any standard Goodness of fit tests for regression trees, where the output is a continuous variable?

I have fitted a regression tree on my training data and would like to demonstrate that it is a good model. For now, I am doing that by calculating the RMSE between the actual values of the dependent ...
Santanu's user avatar
  • 121
1 vote
1 answer
90 views

Goodness of fit test/index for a regression tree

I have fitted a regression tree on my data and would like to demonstrate that it is a good model. Are there any standard goodness of fit test or index for a regression tree? I understand that I can ...
Santanu's user avatar
  • 121
0 votes
1 answer
17 views

Classification criteria equation of decision trees

Can someone explain what does the term I(y=k) stand for in the equation for p_mk ?
Sherwin R's user avatar
2 votes
1 answer
6k views

How to use categorical features in lightGBM? [closed]

I am working on an attrition dataset which has a large number of categorical parameters. Each categorical parameter has a high cardinality, so one-hot encoding them is out of question. I was looking ...
Ashish Samant's user avatar
1 vote
1 answer
61 views

Making a decision tree with numeric data

Working with decision tree and I have couple of questions: Should I always do random forest before or I can just do the decision tree, skip the random forest part? Should I always have a training ...
Sisi's user avatar
  • 61
2 votes
0 answers
40 views

Can you use regression trees for classification tasks in random forest?

I've been playing around with random forest algorithm to classify a binary Y vector using classification and regression trees. Classification trees output class probability and regression trees an ...
Mirko Pavicic's user avatar
3 votes
0 answers
44 views

Exactly when can I model a two-dimensional region with a 100% accuracy decision tree?

This is a theoritical question about decision trees. I believe that the question is very explicit in the title of the question. My thoughts on this question follow below. If the region we are dealing ...
xyz's user avatar
  • 185
2 votes
1 answer
48 views

How to find AUC from Binary Classification Decision Tree?

Decision Tree I have found Misclassification rates for all the leaf nodes. samples = 3635 + 1101 = 4736, class = Cash, misclassification rate = 1101 / 4736 = 0.232. samples = 47436 + 44556 = 91992, ...
Aman Rangapur's user avatar
1 vote
1 answer
72 views

XGBoost: Why is the "approximate algorithm" faster?

I am reading T. Chen, C. Guestrin, "XGBoost: A Scalable Tree Boosting System", 2016 (arXiv), which is seemingly full of typos. They propose the so-called "approximate algorithm" (...
paperskilltrees's user avatar
0 votes
1 answer
22 views

Multivariate analysis for subjective decision making

I am trying to find the best US state using 25 columns of normalized data (best = 1, worst = 0) such as crime rate, GDP, house prices, and others. This results in a 50x25 Excel table. Afterward, each ...
user avatar

1
2 3 4 5
26