Questions tagged [cart]
'Classification And Regression Trees'. CART is a popular data mining technique.
0
votes
0answers
13 views
XGBoost tree dump contains lots of empty trees
After fitting a regression model using XGBoost, I want to inspect the individual trees that were built. In the resulting table, I find a lot of 0-depth trees, i.e. trees with only a leaf node, and ...
3
votes
1answer
32 views
Solve the optimization problem of tree, should we make each rectangle contains exactly one training data point?
I was reading the book "An Introduction to Statistical Learning with Applications in R". In page 306, when talking about the objective function of tree model, the book says:
"The goal is to find ...
1
vote
0answers
12 views
Difference between C5.0 and a 'regular' decision tree
After reading some descriptions of C5.0 trees, it appears to me that it is the same as using e.g. sklearn DecisionTreeClassifier with ...
1
vote
0answers
20 views
Using Regression Trees for Univariate Time Series Data
I have a monthly time series (105 observations) including trend and seasonality and want to forecast the numeric values.
I initially tested with the Box-Jenkins approach and other univariate models ...
4
votes
0answers
38 views
“Hierarchical” Random forests?
Background
I am using Random Forest to classify ~900 objects based on a large number (> 80) predictors. I split these 70:30 for training and testing. The overall model does fairly well, giving an ...
2
votes
1answer
68 views
how does splitting at a node occur in a decision-tree with non-categorical data?
According to a website (:http://dataaspirant.com/2017/01/30/how-decision-tree-algorithm-works/) , these values are chosen randomly:
I don't think this is the case with any optimized solution of a ...
2
votes
2answers
45 views
Why are all predictions made by XGBoost distinct?
If I understood correctly the XGBoost is a framework that operates on gradient tree boosting. It means that behind the scenes, it uses a decision tree to make a prediction. So, from what I read in the ...
1
vote
1answer
16 views
How to estimate the leafsize of the kd-tree?
The kd-tree implementation proposed by the scipy python libray asks for the value of the leafsize parameter that is to say the maximum number of points a node can hold. It is by default set to 10.
...
1
vote
0answers
30 views
Xgboost / Boosted decision trees: Representing categorical id numbers as continuous integer variable
I've been reading through some kernels at kaggle.com for a sales forecasting competition, and noticed that a lot of people using Xgboost are feeding it categorial ID variables, represented as ...
0
votes
0answers
27 views
Understanding a Classification Tree applied to the iris dataset
I used the iris dataset for a simple regression tree to see how things work. I am generally interested in two aspects:
Why does the tree not use the Sepal data to improve the classification
What do ...
1
vote
0answers
22 views
Difference between tree package and manual computation
I have a simple example (only one independent variable)
to do the regression tree using the tree package in R,
which reads
...
0
votes
1answer
13 views
Decision Tree on a set with reliabilty information
I've got an introductory AI course in my university, and I was taught about decision trees. I'm now facing a classification problem that seems solvable with a DT, but I'm stuck with an unseen ...
0
votes
1answer
26 views
How to engineer a bimodal continuous feature for use in Decision Tree?
I have a predictor that exhibits "bimodal" behaviour. How can I engineer this feature to improve performance within a Decision Tree?
For an intuitive example, consider how a binary flag of "moves ...
3
votes
0answers
28 views
Quantifying uncertainty of predictions for new data in the regression tree
I used Regression Learner to train my data. I held out 25% of the input for validation and ran different models for training. Based on the results using RMSE and R-squared, I decided to go for the ...
0
votes
0answers
30 views
Statistical conclusions based on conditional trees
I have a complex dataset, number of features is much bigger than number of samples. The question is - which features are important for classification into 2 groups.
I think that (after some ...
0
votes
1answer
27 views
default plot for mob object (glm tree) not returned; using party package [closed]
I'm trying to plot a glm tree using the package party. Per the reference guide, the default plot of the terminal node should be a spinogram as in this image but I ...
1
vote
0answers
19 views
Classification trees: contingency table and Gini index at split point (grouped categories)
I am working through chapter 14 of the book Applied predictive modeling (Kuhn, Max, and Kjell Johnson. Applied predictive modeling. Vol. 26. New York: Springer, 2013.). For tree models with categorial ...
0
votes
0answers
8 views
Usage of correlated (non-linear correlation) variables in an experiment and standardisation of variables values
The setting (see the dataset at the end of the question)
The setting of the problem is this:
I ask many multiple choice questions;
only 1 out of the 3 available answers is correct;
I ask the same ...
0
votes
1answer
33 views
Repeating CV multiple times
I understood the purpose behind CV (i.e. just to make sure the data is spread well across the folds, and the skewness is somewhat averaged-out).
Let's take this example -
...
1
vote
0answers
52 views
How to interpret feature importance in a decision tree after applying Factor Analysis
I'm using SKlearn to apply Factor Analysis (FA) to my data before training a Decision Tree.
I then want to do an importance analysis. If I had not applied FA to my data, I could just call clf....
0
votes
0answers
14 views
SHAP values vs Information Gain?
SHAP values which are essentially the variable importance at a local level where each variable's importance is assessed by different in probability outcome of a model with and without the variable. I ...
0
votes
1answer
14 views
Shrink decision tree by shuffling order of attributes
I made a decision tree that classifies mushrooms in the UCI Mushroom dataset as either poisonous or edible based on their features. The model has a 100% accuracy on both the training and test set. ...
0
votes
0answers
32 views
ELI5: The logic behind tree building
I'm studying trees for the first time and I got stuck on a theoretical passage: on ISLR (Hastie, James, Tibshirani, Witten) I found that considering that: $$\sum_{j=1}^{J} \sum_{i\in Rj} (y-\hat{y}_{...
0
votes
1answer
14 views
How splits are calculated in Decision tree regression in python
I'm using scikit-learn to build a decision tree (or a random forrest) for a regression problem. I have continuous variables as my regressors. I wonder to know how the splits in a regression decision ...
4
votes
2answers
666 views
Why log-transform to normal distribution for decision trees?
On page 304 of chapter 8 of An Introduction to Statistical Learning with Applications in R (James et al.), the authors say:
We use the Hitters data set to predict a baseball player’s Salary based ...
0
votes
0answers
6 views
CART model: Performance differences depending on preprocessing
I am trying to fit a CART model on a spam dataset, trying to predict "spam" or "ham".
Usually, I work with the "tidy" packages for preprocessing; however, I now learnt about another way in a tutorial,...
0
votes
2answers
53 views
When MSE for CV is greater than test MSE?
In an introduction to statistical learning book, on page 311 there is a figure which compares MSE vs the number of leaves.
I like to know the reason that Cross-Validation's Mean Square Error curve is ...
2
votes
2answers
42 views
Is some degree of overfitting always going to occur in tree based models?
So, I am somewhat new to machine learning, and I am trying my hand at a bunch of different Kaggle datasets. In a lot of the datasets that I ended up a tree-based model on, I noticed one that all of ...
2
votes
0answers
7 views
How to determine whether pruning of decision tree is necessary?
I am really new to decision trees. I constructed classification tree (estimate certification using grade and assignment) and calculated cp:
...
0
votes
0answers
9 views
Evaluating multiclass decision tree as a probability function
I have a problem where I am trying to estimate the transition probability within a state-diagram.
For example, estimating the chances of transitioning from a happy mood to one of 3 possibilities (...
0
votes
0answers
26 views
Incremental/Online Learning for Reinforcement Learning
I have been reading the book Reinforcement Learning: An Introduction by Sutton and Barto, and I have some questions about extending the value function approximation to non-differential functions:
Is ...
1
vote
1answer
36 views
catboost ignoring my holiday category
I'm experimenting with catboost, predicting electricity demand from temperature, timeofday, dayofweek and if a day is a public holiday or not (and a few other continuous and categorical columns but ...
2
votes
0answers
23 views
In Gradient Boosting Tree, why do we fit the tree on the residuals and not on the sum of the previous function and the residuals?
In the Gradient Boosting Tree algorithm, as described in https://en.wikipedia.org/wiki/Gradient_boosting#Gradient_tree_boosting, we update the previous model $F_m$ by adding the results $h_m$ of the ...
3
votes
1answer
104 views
Please correct my assumption on how regression trees work
I'm trying to understand how regression trees work, I've been experimenting with catboost and xgboost in python, and I'm getting results which I don't expect, can someone please clarify (and apologies ...
0
votes
2answers
81 views
Why are decision trees considered supervised learning?
It seems to work similar to clustering algorithms, where data does not have to be labeled, and the algorithm creates it's own labels/groups based on feature similarities...
0
votes
0answers
14 views
How to reduce the number of rules in decision tree with Support and Confidence
The following is a set of rules in decison tree. How to reduce the number of rules in the set with Support and Confidence?
If Ascites = 'Yes' then if Class = 'Live' then if Spiders = 'Yes' then if ...
1
vote
0answers
32 views
Rpart and subsets
I am getting unexpected results from an rpart model, where the model selects two variables, one of which is a subset of the other. This in itself is not unexpected, but the seemingly odd thing is that ...
0
votes
1answer
31 views
How do I use a random forest regression once trained [closed]
Similar questions have been asked on these two posts: random forest how to use the results and How to use random forest for regression after it is trained but I feel like the replies didn't give ...
0
votes
0answers
27 views
How to prevent GBDT from splitting on uninformative features?
I'm looking into using feature importance scores from GBDT for feature selection. Although GBDT does not need manual feature selection, the number of features is a restriction of the production system ...
2
votes
0answers
34 views
Model selection using p-values - tree inference
Suppose I have some i.i.d. normal observations from $\mathbb{R}^f$ with parameters $(\mu, \Sigma)$ and $\Sigma$ is known to be the identity matrix.
I have the following hypotheses:
$H_0^i$: $\mu_i = ...
1
vote
1answer
110 views
Using a decision tree to predict a relevant location to a user
I am trying to create a decision tree to predict new locations a user would like to visit based on previous locations they have liked.
Here is my problem, I have two data sets,
a user data set ...
1
vote
0answers
25 views
Why do RMSE values increase on a smaller tree (RPART)
AIM: I want to understand why does RMSE increase on a smaller tree.
CONTEXT: I am learning the rpart algorithm. I had some data,...
0
votes
1answer
22 views
Using cosine of measurement time as feature for decision trees VS NNs
I have a regression data set and I'm trying to do some feature engineering.
The data set is foot fall coming into a store measured on the hour.
I'd like to include the time of measurement as a ...
11
votes
5answers
2k views
Random Forest and Decision Tree Algorithm
A random forest is a collection of decision trees following the bagging concept. When we move from one decision tree to the next decision tree then how does the information learned by last decision ...
0
votes
0answers
9 views
How to extract and filter rules from C5.0 package for making fuzzy rules
I have project on a ML dataset. I have performed decision tree using C5.0 package in R.
For the next part I want to make fuzzy logic based on C5.0 results. I have 2 questions:
1- How can I filter and ...
0
votes
0answers
5 views
Method for predicting whether a change in survey presentation had significant effect on survey response
I am not experienced in qualitative research in general and so I am hoping to get some guidance on this particular question I am dealing with (and perhaps from there I can extrapolate to some more ...
0
votes
0answers
9 views
What does the attribute splitter in DecisionTreeClassifier do?
The splitter attribute in the constructor of DecisionTreeClassifier(or DecisionTreeRegressor) can take two values 'best' or 'random'. What does the attribute splitter in DecisionTreeClassifier do?
2
votes
1answer
145 views
Gradient Boosted Trees vs Neural Network for limited data [closed]
I have a classification problem, with about 10 different inputs, some boolean, some categorical (and unrelated to each other), some being a float between 0 and 1, which need to be mapped to 4 ...
2
votes
1answer
99 views
Regression tree with nested data repeated in time (GLMERTREE, REEMTREE or REEMCTREE)
I work on the predation of seeds by insects (carabidae), and I am particularly interested in the effect of community composition on predation. I would like to know if the best predation rates are ...
1
vote
0answers
26 views
Boosted decision trees: in which situations are “deep” decision trees performing better?
The general idea of boosted decision trees is to use very simple trees in the following manner (simplified, for intuition only):
start with a simple tree,
fit another simple tree on the residuals,
...
