Cross Validated

Questions tagged [cart]

Ask Question

'Classification And Regression Trees'. CART is a popular machine learning technique, and it forms the basis for techniques like random forests and common implementations of gradient boosting machines.

1,143 questions
Newest
Active
Bountied
Unanswered
Filter by
Sorted by
Tagged with
0
votes
0answers
16 views

Confidence interval for Regression Trees

Many people think the regression tree is only an algorithm and it doesn't make sense approach confidence interval to it so I'd like to know if there's anyone figured out how to do it. A regression ...
0
votes
0answers
9 views

Gradient Boosting vs XGBoost key differences

I know XGBoost minimize a regularized loss function instead GB (gradient boosting) but I dont know how trees grow, it would be a simple fit to estimate G/H? where G is first derivate with respect to ...
0
votes
0answers
17 views

Binary predictive model in R

I want to develop a predictive model based on a binary dependent variable (1 if default, 0 otherwise). Based on methods like logistic regression, decision trees, and random forest in R. My independent ...
1
vote
0answers
16 views

How can I visualize decision tree leaves in tabular form effectively?

I have a decision tree from a classification model that is 6 levels deep and has about 30 different leaf nodes. In a table, I want to sort each leaf node by training probability, and capture ...
0
votes
0answers
6 views

Predicted value smaller than the smallest value in train data in Tree algorithm

I ran the LGBM regressor in Python. I thought that the predicted value (y_hat) from Tree algorithm should be in the range of train data (Y). But I got a smaller predicted value (y_hat) which is ...
0
votes
0answers
6 views

AdaBoost Regression Prediction Inequality

I have created an AdaBoost regression model from scratch in Python but ran into a rare occurrence where a prediction can not be made for an observation in the testing data. I have followed section 3 (...
0
votes
0answers
9 views

`rpart` loss matrix and model comparison by AUC

In logistic regression we first build a model and then eventually we can take into account fact that cost of false negative should be lower/higher than cost of false positive. But in the case of ...
1
vote
1answer
38 views

Regression tree

Self-study question Given $(y_i, x_i)$, $i = 1, . . . , n$, where $y_i \in \mathbb{R}$ and $x_i ∈ R ⊂ \mathbb{R}^p$. Show that $\displaystyle \sum_{i:x_i \in R_1}(y_i − \hat{y}_{R_{1}})^2 +\sum _{i:...
2
votes
1answer
48 views

Does gradient boosted trees actually use regression trees for classification, and if so, what does the gradient update?

I have often read that gradient boosting algorithms fit sequential models to the overall model's residuals, but I can't make sense of this for classification problems (for instance, what is the "...
0
votes
1answer
17 views

How is the threshold parameter practically selected for Scikit learn's decision tree algorithm and how to determine depth of tree?

I am referring to the so-called optimized CART algorithm that is explained on Scikit learn's website: https://scikit-learn.org/stable/modules/tree.html#mathematical-formulation I would appreciate if ...
1
vote
0answers
14 views

Cost complexity pruning decision trees

I am trying to understand cost complexity pruning in classification trees. I found that DecisionTree in sklearn has a function called cost_complexity_pruning_path, which gives the effective alphas of ...
1
vote
1answer
28 views

Two question about a decision tree algorithm I found online

I am trying to learn decision trees but it has been difficult because the examples are extremely long and tedious and everybody seems to have a different algorithm in mind After some digging I found ...
1
vote
0answers
16 views

Issue poisson method rpart

I tried to use the rpart package to deal with observations count of crop pests, but I realized that there was overfitting with poisson method. In the vignette of the package, it said that it is ...
0
votes
1answer
15 views

How do decision trees decide the value to be split upon for continous variables? [duplicate]

I know that decision trees make the split based on some metric such as entropy, information gain, gini index etc. But for continous variables how does it figure the value at which to make a split. For ...
0
votes
0answers
12 views

question about time complexity of split finding for Column Block in xgboost

I read the Xgboost paper and I have several questions in the 4.1 section Column Block for Parallel Learning 1.the third paragraph of which says The block structure also helps when using the ...
0
votes
0answers
12 views

Maximum number of leaves in top-down binary decision tree (CART/C4.5)

Out of curiosity, I was wondering if there are any theoretical bounds/guarantees on the number of leaves that a classification decision tree built using an impurity-based algorithm (such as CART or C4....
0
votes
0answers
9 views

Is a regression tree closed under scaling?

I've tried proof that a regression tree is closed under scaling but I'm not sure if I've meant right. Please go through. Let $D^n$ be a feature space, a regression tree can be viewed as function that $...
0
votes
0answers
8 views

Is conditional inference tree (ctree) scale and location invariant?

I know that classical approaches for decision tree building where splits rely on the information metrics (such as CART using Gini) are scale and location invariant, i.e., one does not have to ...
0
votes
0answers
15 views

How are these definitions of deviance related?

My questions concerns the deviance in the context of CART. In Elements of Statistical Learning II (p. 309), deviance at node $m$ is defined as $-\sum_{k=1}^K\hat{p}_{mk}\mathrm{log}\hat{p}_{mk}$, ...
0
votes
0answers
13 views

Decision tree as function

How should i formalize regression tree as $f: C \rightarrow R$ ? Where C is any feature space (not necessarily $\Re^p$ where $p$ is a dimension of an element in $C$)
0
votes
0answers
15 views

Example of scenario where first split does not improve gini impurity

Suppose one uses gini impurity to find the best split while constructing the classification tree. Give an example of a scenario where the first best split does not improve the gini impurity compared ...
0
votes
1answer
52 views

Using a Bayesian Additive Regression Trees model for causal inference

Some Context: I've read this presentation about using a BART model to find out the causal effect of a certain variable with respect to a target variable (say, how much does a specific medicine ...
0
votes
1answer
11 views

Intuition behind training and test MSE when using regression trees

Imagine that we have a supervised learning setting. Training data is given by the input-output pairs $(\mathbf{x}_n, y_n)$ for $n=1,\dotsc,N$ and similarly, the test data $(\mathbf{x'}_n, y'_n)$ for $...
0
votes
0answers
10 views

MSE decomposition of tree estimator

In this paper, a tree is defined as a partitioning $\Pi = \{ \ell_1,\dots, \ell_N\}$ of the feature space $\mathcal{X}$, and given a specific partitioning $\Pi$, the conditional mean function is: $$ \...
1
vote
2answers
91 views

AdaBoost - why decision stumps instead of trees?

Since the original AdaBoost article it has been found out that boosting reduces both variance and bias in the classifier (in contrast to bagging, which only reduces variance). Original AdaBoost (and ...
1
vote
1answer
31 views

Is training a tree faster with squared-error loss as compared to other losses?

I am reading through various introductions to decision trees and boosting algorithms. It strikes me that people always like to resort to squared-error loss whenever a tree needs to be trained. For ...
0
votes
0answers
31 views

What are the disadvantages of one hot encoding in trees?

As Sklearn need to encode categorical features in order to run tree based algorithm, i was wondering what are the fact i should be careful when analysnig the outputs (predictions, feature importance ...
0
votes
0answers
8 views

How would we build a non-greedy Tree?

I understand that for Cart Tree, given we are at a specific node we have to select the "best" combination (feature,value of feature) in the sense of a purity measure. It is a local optimal ...
2
votes
1answer
41 views

Why permuting a predictor gives a measure of the importance of the variable?

I am reading the vignette for the R package randomForestExplainer. This package allows us the compute the importance of variables in a random forest model. The result of the function ...
2
votes
1answer
28 views

What does it mean to “permute” a predictor in the context of random forest?

I am reading the vignette for the R package randomForestExplainer. There the result accuracy_decrease (classification) is defined as mean decrease of prediction ...
1
vote
1answer
18 views

Bagged Decision trees / Random Forests: why ISLR uses validation set instead of OOB to compute out-of-sample MSE?

I am reading the book "An Introduction to Statistical Learning" available here. Chapter 8.3.3 at page 328 of the book computes a bagged decision tree (which is a random forest where we use ...
0
votes
0answers
16 views

How to measure ML classification model stability?

It's well known that decision trees are unstable models, i.e. they have high variance. It can be easily shown by adding Gaussian noise to variable(s) and checking that completely different tree is ...
4
votes
1answer
55 views

Why Gradient Boosting is needed at all?

I am trying to learn from Hastie on boosting methods. A question has bothered me for weeks. The book describes Forward-Stagewise Fitting, that is, fit a weak learner to explain the residuals from the ...
0
votes
0answers
8 views

decision tree yields identical predicted value

I am reading ISL page 314 about tree-based classification. I don't quite understand the author's explanation of why some of the splits can "yield two terminal nodes that have the same predicted ...
0
votes
0answers
9 views

How can I force sklearn decision tree to select predefined feature as the root?

I would like to create a single decision tree from my data, but I want the decision tree to make the first split using a feature that I selected, not necessarily the feature that creates the best ...
1
vote
1answer
57 views

Which assumptions should be checked for regression tree to validated model?

I am working with regression tree. I have four predictors. There is a exponential relationship between predictor and dependent variable. But after building predictive model I cannot understand whether ...
0
votes
0answers
10 views

How can I generate a plot of the partitions in Isolation Forests

I have seen this plot is used to indicated how anomalies are isolated via partitioning in Isolation Forests. Is there a library to automatically plot this from a dataset? The plot I want to generate ...
1
vote
0answers
10 views

How to calculate the number of possible decision trees of given feature-target set?

Is there a method to calculate the search space of decision trees for different depth values? Let's assume we have 5 binary feature and 1 binary target. How can we calculate that how many possible ...
0
votes
0answers
18 views

How variable importance for decision tree classifier in `caret` is estimated?

I trained a decision tree classifier by means the package caret, This is the code: ...
0
votes
1answer
17 views

Different accuracies for different algorithms

I have a binary classification dataset with all features in numeric form. Most of them are continuous variable (within ranges, for example - from 2.50001 to 2.9999). I want to predict the output which ...
0
votes
0answers
16 views

Decision tree learning curve - training accuracy does not decrease with increase in training set size

I am training a Decision Tree classifier. I was initially happy with the performance metrics of the model, but I wanted to plot the learning curve to get a better understanding of how my model is ...
1
vote
0answers
15 views

How can I show a mathematical proof of entropy in clasification tree? [closed]

I am trying to understand the splitting criteria in the classification tree. How can I show that for $p_1,p_2,..,p_n$ these functions attaining their maximum and minimum? $g(p_1,p_2,...,p_n) = Σp_i(1-...
1
vote
1answer
29 views

Why the prediction of this Random Forrest model is so poor?

I am using Random Forrest to predict the MRR (Material removal rate). But the predictions have been quite off the mark. Even Linear Regression gave a much better result. I don't know where I'm going ...
1
vote
0answers
19 views

Using multiple probability cutoffs for a logistic regression model?

Have data with "valid" and "invalid" classes, lots of predictors, over 15. Only 5% of data set is valid (success class 1), 95% is invalid class 0. The number of invalids is skewing ...
1
vote
1answer
191 views

Optimise Random Forest Model using GridSearchCV in Python

I am working on a classification problem where I am applying various machine learning models. I have used DecisionTreeClassifier from Sklearn on my dataset using the following steps: Calculated alpha ...
0
votes
0answers
20 views

How to address severe class imbalance with lots of zeros in the columns? [duplicate]

My datasets has severe class imbalance with lots of zeros in the columns. Here is the total count of my samples. Total Samples: 12237697 Positive samples: 1061 (0.01% of total) I have tried weighted ...
1
vote
0answers
22 views

How to parametrize a Decisiontree Classifier for NLP tasks?

I want to use sklearns DecisionTreeClassifier for sentiment analysis. I am fully aware that a decision tree is probably not a good model for this kind of task, but I want to try it anyways for the ...
0
votes
0answers
11 views

Internal validation of BART using bootstrap resampling

I developed a BART model for binary outcome predictions given two numeric independent variables using the pbart R package (v2.9). The code looks like this (sub-setting for 23 observations, complete ...
2
votes
0answers
26 views

Using cross-validation for model selection and model comparison

Let us suppose that we have two classifiers: SVM and CART. For each one of them, a set of hyperparameters is considered (C=0.001,0.01,... for SVM cp=... for CART). The question is, can I use k-fold ...

1
2 3 4 5
…
23

Your privacy

By clicking “Accept all cookies”, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy.