Questions tagged [cart]
'Classification And Regression Trees'. CART is a popular machine learning technique, and it forms the basis for techniques like random forests and common implementations of gradient boosting machines.
992
questions
0
votes
1answer
20 views
What does it mean to predict “with a constant” in a decision tree?
I'm new to this exchange and ML. Below is a snippet (hopefully it's enough context) from The Elements of Statistical Learning by Hastie, Tibshirani, Friedman:
We first split the space into
two ...
0
votes
0answers
12 views
Assigning different COST for each event in a classification model
I need some help with a project (described below) for a banking client. Any advice/suggestions would be greatly appreciated.
Project :
We are trying to model credit card attritions. The event rate ...
0
votes
0answers
16 views
Why can a Random Forest generate extreme values?
I'm training a Random Forest Regressor, and I'm using a target variable thats values are between negative and positive 40. The regressor performs relatively well, however, I've noticed sometimes it ...
1
vote
2answers
54 views
Is there a guide for when to implement time series techniques?
I am interested in getting a better sense as to when to use time series techniques.
Let's say you have a data set with units sold as the response. Your goal is to predict units sold on any given ...
0
votes
0answers
31 views
Why use this Cost-function for pruning in Decision Trees?
So, I've been learning about decision trees and weakest link pruning for regression from ISL and ESL. But there is a couple of things that are still unclear:
We use the RSS (residual sum of squares) +...
1
vote
0answers
28 views
Decision tree- Alternative model to predict this data?
My data looks something this (for example):
...
0
votes
1answer
10 views
Regression trees with geepack package
I can’t run regression trees without geeglm. I have longitudinal data so rpart wouldn’t work. Is there a way to get regression trees with geeglm?
0
votes
1answer
36 views
What is minimized/optimized when we use AdaBoost
When I learned about CART, we learned that at each split, we try to minimize some measure (usually Gini index) of the split. That is, we determine the predictor and threshold that decreases the Gini ...
2
votes
0answers
31 views
Why gradient boosting uses sampling without replacement?
In Random Forest each tree is built selecting a sample with replacement (bootstrap). And I assumed that Gradient Boosting's trees were selected with the same sampling technique. (@BenReiniger ...
0
votes
0answers
8 views
Is it possible to do cross-validation with GEE?
Is it possible to do cross-validation with GEE and recursive partitioning trees?
I wanted to use recursive partitioning on longitudinal data and them run the data through gee.
0
votes
0answers
3 views
Are decision trees useful for multi-label classification with non numerical/comparable categories?
I know decision trees can be useful for multi-label classification. For instance with the iris dataset:
...
1
vote
1answer
15 views
SMOTE in decision tree is generating a “Synthetic” rule
I am running a decision tree and to balance the class labels I used SMOTE. The dataset originally consisted of 350k records and after the balancing is 1.400k records, and the resultant decision tree ...
0
votes
0answers
16 views
Decision trees minimizing the gini error
I was reading the element of statistical learning and I stumble upon the formula for minimizing the misclassification error. I was wondering if I could write something like that for the Gini Index.
...
1
vote
1answer
64 views
Is there an ExtraTreesClassifier-like classifier that has decision boundary function like SVM?
I'm using sklearn and I tested many models and those two worked best: Linear SVM and the ExtraTreesClassifier as binary classifiers. The ExtraTreesClassifier outperforms the Linear SVM in terms of ...
1
vote
1answer
34 views
What is the difference between Gini index and Gini coefficient?
I am building a decision tree from scratch. I have been using entropy so far (calculated this way):
...
1
vote
1answer
40 views
Can someone explain to the Gini Index for a tree?
So I know what the formula for the Gini index. However, I have a few questions that I am hoping to clarify. I saw this, which tells you how to calculate the Gini index for each feature: Computing ...
2
votes
1answer
56 views
Finding the optimal split threshold for a feature using XGBoost in R
While implementing a gradient-boosted tree algorithm on a dataset, is it possible to learn the optimal value for a feature that best predicts a class?
For example: In the iris dataset, what is the ...
0
votes
0answers
23 views
Gains on test data set higher than that on training data set post balancing
I have an imbalanced data set (96-4 split between No and Yes cases). I am trying to build a decision tree model for classification after balancing my data set(tried different thresholds for ...
1
vote
1answer
30 views
The accumulative tree structure in a tree based gradient boosting
I'm playing with gradient boosting methods and with its python packages out there. I tried lightgbm, started with a high-dimensional input to predict a task. A left ...
0
votes
1answer
14 views
What is the default node splitting process carried by sci-kit's RandomForestRegressor when all features and target are continuous?
I have some data containing several features, mainly continuous variables.
Implementing the randomForestRegressor algorithm from the sci-kit package in Python is relatively simple and results look OK....
2
votes
1answer
60 views
Random Forest pruning vs stopping criteria
I have recently noticed that SciKit-Learn now supports Cost Complexity Pruning, which is great. Since this has been implemented, should I still use other regression trees/ random forest hyper-...
1
vote
0answers
17 views
Decision trees: maximizing information gain vs. minimizing conditional entropy?
Information gain is defined as $$IG(T, a) = H(T) - H(T|a),$$ where $H(T|a)$ is the conditional entropy of $T$ given attribute $a$, and $H(T)$ is the prior entropy of our dataset before we test out the ...
1
vote
0answers
40 views
Building a Classification model for predicting Customer Churn
I am currently building a Customer Churn Prediction model and the project is in the process of development of models. The client has given data till Sep 2019 and wants to check if the model is able to ...
0
votes
0answers
22 views
Faster version of mobForest for Linear model trees?
I have a data set that I want to fit a linear model tree to it by tree partitioning on certain variables and linear fitting on the leaves for the remaining variables. I see mobForest has an ...
1
vote
0answers
50 views
How can I preprocess my data better? Help finding issues with the Dataset to improve accurracy
The 'DATA' and my Jupyter notebooks can be found here
Now my issue is that for my data-set I get an mean accuracy from grid search of 0.64 (I get different for other models, like for svm 0.75) but ...
1
vote
0answers
13 views
Regression trees and choosing threshold values to minimize mean squared error
I'm trying to learn a decision tree for a regression problem. Each node of the generated tree, will split by the criterion variable < threshold for some ...
1
vote
0answers
34 views
ARFF dataset. Did I train my decision tree model correctly?
I am new to ML and coding so I thought a fun first project would be to create a decision tree that can detect phishing links.
I plan to use two datasets. This is the first one which I am currently ...
0
votes
0answers
9 views
Could someone please give an concrete example to illustrate what the formation of regions is?
Page 666 in pattern recognition and machine learning (free)
gives these two formulas
and says
They
encourage the formation of regions in which a high proportion of the data points are
...
1
vote
1answer
72 views
When re-fitting XGBoost on most important features only, their (relative) feature importances change
I am using 60 obseravation*90features data (all continuous variables) and the response variable is also continuous. These 90 features are highly correlated and some of them might be redundant.
I am ...
1
vote
1answer
19 views
Complexity associated with decision trees
According to the sklearn documentation on decision trees:
The cost of using the tree (i.e., predicting data) is logarithmic in the number of data points used to train the tree.
Could somebody ...
0
votes
1answer
12 views
Does “adding nodes one” mean “adding a layer of nodes”?
section 14.4 in pattern recognition and machine learning (free) says
Now consider how to determine the structure of the decision tree. Even for a
fixed number of nodes in the tree, the problem of ...
1
vote
1answer
29 views
Could someone please gives an concrete example to illustrate what a **predictive variable** is in decision tree learning?
section 14.4 in pattern recognition and machine learning (free) gives this figure
and says
Within each region, there is a separate model to predict the target variable. For
instance, in ...
0
votes
1answer
8 views
Is it reasonable to view each leaf node of a decision as a simple sub-model associated with a distinct class label?
section 14.4 in pattern recognition and machine learning (free) says
There are various simple, but widely used, models that work by partitioning the
input space into cuboid regions, whose edges ...
1
vote
0answers
49 views
Scikit learn Decision Tree not deterministic [closed]
I am doing recursive feature elimination and cross-validated selection (RFECV) in order to get the best number of features. As I will be comparing different hyper-parameters and methods in dealing ...
1
vote
1answer
17 views
How does decision tree decide which variable to use in next split?
The CART (or RPART) algorithm uses gini index to find a threshold value for a variable in each split. But how does it choose which variable will it use for splitting ?
0
votes
0answers
12 views
Decisions Tree - JOINT Probability rpart
Please describe if I could calculate joined probability for No class. I am using R and rpart. I want to check how probability will change if I make cuts under my tree like attached.
73%*34% + 71%*22%
0
votes
0answers
21 views
Creating a regression tree with a specific number of terminal nodes that minimize RSS
I am currently learning tree, but I have some issues understanding it.
I would like to draw a tree with 4 terminal nodes for 5 observations that minimize the RSS. How could I do that manually or ...
1
vote
1answer
14 views
Interpreting a classification tree
I am currently trying to understand how I should interpret my classification tree. This is mine :
How can I know which group most likely to swipe left, the probability of swiping left and the ...
1
vote
0answers
18 views
What factors will make a decision tree superior to a logistics regression model, taking into account the following? [closed]
Assume that the lift chart indicates that the decision tree is inferior, the C statistic is superior for the logistics regression also.
What other factors will make a decision tree better, even in ...
0
votes
0answers
22 views
Decision tree with weights trained using RandomizedSearchCV - do I have to refit?
I trained a decision tree with weights using RandomizedSearchCV:
...
0
votes
0answers
7 views
Should I prune the tree trained on parameters obtained by Bayesian Optimization?
If the parameters for a decision tree are obtained by bayesian optimization, where the 10-fold cross validation error is minimized (or 1- error maximized) and afterwards the model is trained on the ...
0
votes
0answers
28 views
Should I use stratify parameter for scikit-learn train_test_split while dealing with highly unbalanced dataset?
I have a dataset with over 200000 records. Only 400 of are positive, which makes the data highly unbalanced. I cannot collect more data.
At first I trained a decision tree. I used StratifiedKFold and ...
1
vote
0answers
58 views
Can someone explain Ripley's proof (1996) of the splitting of categorical variables?
Specifically I am referring to this theorem:
1) Suppose there are two classes. For a categorical feature , order
the levels in increasing $p(1\mid x = x_i)$. Then a split of the form
$\{x_1, ... x_{\...
0
votes
1answer
16 views
How can we include hourly traffic series data in the rows of train data set for training?
I have a classification problem where I am planning to use hourly traffic data for a day. Is there any way to compress it? instead of creating 24 predictors which account for hourly traffic?
1
vote
2answers
44 views
Does it make sense to use bayesian optimization for tuning of hyperparameters of decision tree model?
Does it make sense to use bayesian optimization for tuning of hyperparameters of decision tree model? I have not found any article or anything related to this, as BO is usually used for black-box ...
0
votes
0answers
23 views
Does random forest (and, decision tree) require an independent observation assumption? [duplicate]
I am wondering if random forest models require an independent observation assumption.
My date includes observations from the same participants, but I do not have a way to identify each participant. ...
4
votes
0answers
26 views
How is EMSE derived for causal trees in Athey and Imbens (PNAS 2016)?
Athey and Imbens build a non-parametric matching procedure to identify and estimate causal effects. To this end, they minimize the expected mean squared error (EMSE) of their procedure, but I don't ...
1
vote
1answer
18 views
Experimental data segmentation using trees based on means - could single trial estimates improve reliability?
I have some categorical data and measures of participants accuracy. Let's say that it is a quiz and we have 8 different categories: History, Geography, Physics and so forth. Each participant is ...
0
votes
0answers
6 views
Prevalence outcome with conditional trees
I'm creating a Conditional Tree model to predict the point prevalence of a certain event estimated in some health centers.
I thought of using the proportional prevalence [0,1] as the outcome, but the ...
0
votes
1answer
64 views
Can a decision Tree split on a question with two features?
If I have two classes that are cleanly separated by a single diagonal line as shown below, can I have a depth 1 decision tree using both features on the question x_2 > x_1 or can we not use two ...
