Questions tagged [cart]
'Classification And Regression Trees'. CART is a popular data mining technique.
0
votes
0answers
4 views
Adding probabilistic suffix trees
In the context of Variable Length Markov Chains, I am interested in fitting VLMCs to longitudinal measurements of a set of patients without pruning and then add the trees together. Then, I could prune ...
0
votes
0answers
5 views
How to infer results from tree-based feature selection and chi squared?
I have a data set with 12 continuous features, with 3 discrete output labels. I want to determine the two best features.
My research thus far has led me to use chi^2 tests and extra tree classifiers ...
0
votes
0answers
21 views
When will a decision tree outperform a Random Forest
I have been comparing a range of tree based machine learning regressors:
Decision Tree
Random Forest
AdaBoost
For each of these regressors 10 fold cross validation has been performed, and 200 ...
0
votes
0answers
21 views
Is it useful to compute R Squared for regression trees? [duplicate]
I have a regression tree and want to validate the peformance. The first measure I have is the mse to find which model is the best. After that I want to check if the model peforms better then an ...
5
votes
1answer
49 views
Should OOB (Out Of Bag) error be less than a Test set error in Random Forests?
I am using the book, "An introduction to statistical learning with applications in R" and reading the section on using OOB to estimate the model error for Random Forests. The graph seems to suggest ...
0
votes
1answer
37 views
Can regression trees outperform classification trees?
I'm learning about trees in my data science class recently and I know that the convention is to use regression trees when the response is continuous and classification trees when the response is ...
0
votes
0answers
16 views
Variance-bias trade off in classification and regression trees
I know what the variance/bias trade off is when I am talking about regression problems. Also in this context i understand the technical derivation.
But I don't have an idea of the variance-bias trade ...
0
votes
0answers
17 views
Using decision tree + active learning for regression?
Existing literatures that concerns using decision tree to do regression is more limited compared to its classification companion. The same also holds for research regarding active learning. I am just ...
0
votes
1answer
42 views
Which algorithm does Decision Tree classifier in sklearn implement?
Which algorithm does Decision Tree classifier in sklearn Library implement? Is it GUIDE?
There are a total of 6 techniques available according to my knowledge, according to this paper
0
votes
0answers
18 views
What is the meaning of “extremity of values for regression” when using sklearn.tree.export_graphviz?
I'm using sklearn.ensemble.RandomForestRegressor, and I would like to show the decision tree of one estimator using sklearn.tree.export_graphviz, but I don't understand what the meaning of extremity ...
1
vote
1answer
25 views
Do decision tree's perform variable selection?
I'm a bit confused how decision tree's select the variables to split.
I know they splitt the data set through variable to get a more pure data set.
But can it happend that some explenatory variables ...
0
votes
0answers
13 views
Why does Weka output decision tree with multiple children nodes of the same target variable?
I'm working with this dataset. I broke the quality class into 3 categories: low, medium and <...
1
vote
0answers
28 views
Wrong calculation of feature importance of decision tree in R [closed]
I trained decision tree both in python and R, but I think the way feature importance is calculated in R may be wrong. Following is the sample code which you can use to reproduce the problem. Let's say ...
1
vote
2answers
30 views
Can a decision tree make a decision based on two variables at one split?
I know that the random forest algorithm works by generating a set of decision trees with a subset of features.
Say I was using random forest as a classification algorithm looking at someone's data ...
-1
votes
0answers
42 views
Unbalanced classes multiclass
Certain ML algorithms have parameters which can be used to deal with the effects of unbalanced dependent variable classes.
For example the random forest implementation in Sci-kit learn has the class ...
2
votes
0answers
12 views
How should I configure CART for easier interpretation?
My goal is not to get better accuracy, but to make interpretation easier and have more stable splits.
So far most useful is use larger min_samples_leaf, but I always get nodes size almost same as ...
2
votes
3answers
118 views
What are the advantages of Random Forest over Decision Trees
I am currently searching for the advantages of random forest over decision trees, but unfortunately I didn't find a research paper that does such a conclusion that summarize all the advantages of RF ...
1
vote
0answers
23 views
Why we get different important factors when performing logistic regression and decision tree?
Why we get different important factors when performing logistic regression and decision tree? When I select features, I perform logistic regression and decision tree separately and wanna see the ...
0
votes
0answers
23 views
Difference in model performance measures of train and test data sets
I am using CART classification technique by dividing a dataset into train and test sets. I have been using Mis-classification error, KS by rank ordering, AUC and Gini as MPMs(model performance ...
2
votes
1answer
30 views
Formal Definition of Tree-Consistent
I am working my way through this paper on Bayesian Hierarchical Clustering. I keep seeing the phrase tree-consistent. However, it doesn't seem to be defined anywhere in the paper. There is a ...
1
vote
0answers
16 views
How do I reconcile the output of a decision tree visual vs the output of the DecisionTreeClassifier class?
I have a decision tree model stored in dt. I also plotted and can visually see the decision tree itself. Given a new, unseen customer, I can use the visual tree to ...
0
votes
0answers
10 views
what does the DeltaCriterionDecisionSplit and NumPredictorSplit in CompactTreeBagger (Matlab)means?
when I try to use CompactTreeBagger in Matlab, I do not know what does the DeltaCriterionDecisionSplit and NumPredictorSplit means. Even though I search the offical guidence https://ww2.mathworks.cn/...
1
vote
0answers
19 views
What is the connection of correlation between variables to decision tree feature importance
I am using decision tree regression to predict a variable.
I realized that the highest correlating features to my target variable are not the ones I get returned with a high feature importance.
...
1
vote
1answer
30 views
Updating decision tree for new data
Lets say you have trained a decision tree for 40 gigs of data.
On Monday morning you receive 10Gig new data and produce some results quickly
to report to your boss.
Can you update the decision tree ...
0
votes
1answer
15 views
Can we predict the monthly sales amount of the coming month without knowing the values of the independent variables of the coming month
I have a data set where the monthly sales of TMT bars and various other explanatory variables are present from April 2014-March 2018. I need to predict the monthly sales of the coming/next month.
...
4
votes
2answers
135 views
Is it ever recommended to use mean/multiple imputation when using tree-based predictive models?
Everytime that I am making some predictive model and I have missing data I impute categorical variables with something like "UNKNOWN" and numerical variables with some absurd number that will never be ...
1
vote
0answers
28 views
Can CART models be used to select features for a logistic regression?
Can I use the features selected from the CART(Classification and Regression Trees) model and take those features and then model the logistic regression using those selected features? Then interpret ...
0
votes
1answer
16 views
How does the exhaustive search for training decision trees work?
I am reading Pattern Matching and Machine Learning by Bishop at the moment. The chapter about decision trees states that to train a decision tree, one can use a greedy algorithm. The book describes ...
2
votes
0answers
34 views
Accuracy and ROC for Logistic and Decision Tree
So I run a logistic regression and decision tree model using same data. The accuracy shows that the decision tree outperforms logistic slightly. However, my ROC curve shows that logistic is much ...
0
votes
0answers
6 views
Incorporating seasonality into Chaid models
I am trying to predict no. of policies sold by a particular insurance agents in a particular month. The problem with Insurance slaes is the sales peaks up in the jan,Feb, MArch month (due to tax ...
0
votes
0answers
26 views
Trouble with size of C5.0 tree in R
I have been working on a fraud detection rules-based system. I have found C5.0 in R to have the most predictive power, but now I am running into an issue. I am only submitting around 12 variables to ...
0
votes
0answers
12 views
Decision tree model selection process
Trying to apply train, validation, and test set to a Binary Decision Tree classifier, following the logic of https://www.coursera.org/learn/machine-learning/home/welcome.
The logic is as follows, if ...
1
vote
0answers
25 views
XGBoost tree dump contains lots of empty trees
After fitting a regression model using XGBoost, I want to inspect the individual trees that were built. In the resulting table, I find a lot of 0-depth trees, i.e. trees with only a leaf node, and ...
3
votes
1answer
35 views
Solve the optimization problem of tree, should we make each rectangle contains exactly one training data point?
I was reading the book "An Introduction to Statistical Learning with Applications in R". In page 306, when talking about the objective function of tree model, the book says:
"The goal is to find ...
1
vote
0answers
28 views
Difference between C5.0 and a 'regular' decision tree
After reading some descriptions of C5.0 trees, it appears to me that it is the same as using e.g. sklearn DecisionTreeClassifier with ...
1
vote
1answer
43 views
Using Regression Trees for Univariate Time Series Data
I have a monthly time series (105 observations) including trend and seasonality and want to forecast the numeric values.
I initially tested with the Box-Jenkins approach and other univariate models ...
4
votes
0answers
85 views
“Hierarchical” Random forests?
Background
I am using Random Forest to classify ~900 objects based on a large number (> 80) predictors. I split these 70:30 for training and testing. The overall model does fairly well, giving an ...
3
votes
1answer
107 views
how does splitting at a node occur in a decision-tree with non-categorical data?
According to a website (:http://dataaspirant.com/2017/01/30/how-decision-tree-algorithm-works/) , these values are chosen randomly for both gini index and Entropy method:
I don't think this is the ...
2
votes
2answers
54 views
Why are all predictions made by XGBoost distinct?
If I understood correctly the XGBoost is a framework that operates on gradient tree boosting. It means that behind the scenes, it uses a decision tree to make a prediction. So, from what I read in the ...
1
vote
1answer
20 views
How to estimate the leafsize of the kd-tree?
The kd-tree implementation proposed by the scipy python libray asks for the value of the leafsize parameter that is to say the maximum number of points a node can hold. It is by default set to 10.
...
1
vote
0answers
52 views
Xgboost / Boosted decision trees: Representing categorical id numbers as continuous integer variable
I've been reading through some kernels at kaggle.com for a sales forecasting competition, and noticed that a lot of people using Xgboost are feeding it categorial ID variables, represented as ...
0
votes
0answers
47 views
Understanding a Classification Tree applied to the iris dataset
I used the iris dataset for a simple regression tree to see how things work. I am generally interested in two aspects:
Why does the tree not use the Sepal data to improve the classification
What do ...
1
vote
0answers
25 views
Difference between tree package and manual computation
I have a simple example (only one independent variable)
to do the regression tree using the tree package in R,
which reads
...
0
votes
1answer
15 views
Decision Tree on a set with reliabilty information
I've got an introductory AI course in my university, and I was taught about decision trees. I'm now facing a classification problem that seems solvable with a DT, but I'm stuck with an unseen ...
0
votes
1answer
77 views
How to engineer a bimodal continuous feature for use in Decision Tree?
I have a predictor that exhibits "bimodal" behaviour. How can I engineer this feature to improve performance within a Decision Tree?
For an intuitive example, consider how a binary flag of "moves ...
3
votes
0answers
32 views
Quantifying uncertainty of predictions for new data in the regression tree
I used Regression Learner to train my data. I held out 25% of the input for validation and ran different models for training. Based on the results using RMSE and R-squared, I decided to go for the ...
0
votes
0answers
32 views
Statistical conclusions based on conditional trees
I have a complex dataset, number of features is much bigger than number of samples. The question is - which features are important for classification into 2 groups.
I think that (after some ...
0
votes
1answer
50 views
default plot for mob object (glm tree) not returned; using party package [closed]
I'm trying to plot a glm tree using the package party. Per the reference guide, the default plot of the terminal node should be a spinogram as in this image but I ...
1
vote
0answers
24 views
Classification trees: contingency table and Gini index at split point (grouped categories)
I am working through chapter 14 of the book Applied predictive modeling (Kuhn, Max, and Kjell Johnson. Applied predictive modeling. Vol. 26. New York: Springer, 2013.). For tree models with categorial ...
0
votes
0answers
9 views
Usage of correlated (non-linear correlation) variables in an experiment and standardisation of variables values
The setting (see the dataset at the end of the question)
The setting of the problem is this:
I ask many multiple choice questions;
only 1 out of the 3 available answers is correct;
I ask the same ...
