Questions tagged [cart]
'Classification And Regression Trees'. CART is a popular data mining technique.
0
votes
0answers
12 views
How the probability in a decision tree is calculated? [closed]
I build a classification tree and predicted the probability that an observation belong to the positive class. Now i ask myself how this probability is calculated?
1
vote
0answers
14 views
Decision Trees - Regression trees weighting of child nodes?
I'm familiar with how classification trees weight the impurity measure of a potential split by the proportion of observations that would fall into each child node, such as:
$$
loss = \frac{n_1}{N_m} ...
0
votes
0answers
38 views
Gini Index Formula
I've read many related articles and posts. The more I read, the more I got confused about 'Gini index' and 'Gini Impurity'. I understood the concept but it seems to me that these things are used ...
0
votes
0answers
27 views
Gradient Boosting regression model produces inaccurate forecasts in presence of outliers in a training set [closed]
I'm using XGBoost with a 'reg:linear' objective in order to forecast time series one step forward.
I notice that the model appear to start producing forecasts that are too high when there were high-...
1
vote
1answer
16 views
Decision tree when leaf nodes are numerical values
I'm having a dataset, where I created a decision tree. My output variables are binary values. I selected two features and the decision tree was generated using R. This is my code.
...
0
votes
1answer
15 views
How to deal with repeated words in training and testing set using CART?
I have a dataset where a child read a passage out loud. Each row is one word (I’m order of the passage). I have a 1/0 for whether they read the word incorrectly or correctly. I’m trying to predict ...
0
votes
0answers
3 views
Relationship between CART single node tree, variable correlation and variable variance
I am currently working on finding a pattern of when CART produces single node trees after cost-complexity pruning. I am most interested in the effect of variable variance and variable correlation. ...
0
votes
0answers
30 views
Are feature importances from tree based models directly actionable for business?
If my response variable say is "has_repurchased" [0 or 1] and I have all customer level features. Can I rank the features in order of importance from the random forest model and report them as whats ...
2
votes
1answer
55 views
Can balanced accuracy be higher than accuracy?
I have classification tree where the balanced accuracy of the test set is higher than the normal accuracy. I thought balanced accuracy can only have at his maximum the same value as the accuracy not ...
2
votes
2answers
57 views
How to check if i have strong linear relationship between dependent variable and independent variables in linear regression (OLS)?
I want compare the out of sample prediction from an linear regression model (OLS) and a regression tree. I read that OLS outperforms regression tree if the relationship between the dependent variable ...
0
votes
1answer
13 views
Calculating a continuous variable with regression trees
I have sample records with several attributes (predictors) and a predicted variable Yes/No.
What I need is, given new data that omits the column Yes/No, to know what is the probability of Yes. Note ...
0
votes
2answers
27 views
regression tree vs linear regression
I'm using one explanatory variable in a regression tree and in a linear regression. The tree finds a split (with variance reduction splitting rule), though R2 is pretty small (0.2). On the validation ...
17
votes
3answers
2k views
Is random forest for regression a 'true' regression?
Random forests are used for regression. However, from what I understand, they assign an average target value at each leaf. Since there are only limited leaves in each tree, there are only specific ...
0
votes
1answer
56 views
Performance Imbalance Dataset Decision Tree
I have a imbalance dataset for a classification task, with the minority class accounting for about 21% of the total.
When I use a decision tree based model for prediction, let's say a classification ...
1
vote
4answers
75 views
How regression trees split, when all the Features and target have only continuous values
Can anyone please explain how splitting is performed in regression trees when we only have continuous features. I have referred to different papers, but all I could find is formulas or theorems.
Can ...
0
votes
0answers
9 views
conditional trees unexpected grouping — differences with chi squared test
I grouped some data with ctree (party package, conditional trees) (binary target) and the result is:
...
2
votes
2answers
40 views
R result interpretation conditional inference tree result for nominal response
I have nominal responses, "yes/no/don't know", that I am using in a conditional inference tree in R. I am having trouble with how to interpret the model's output concerning one of the independent ...
0
votes
0answers
23 views
What is the objective function in cost-complexity pruning in rpart for classification tree?
I constructed a classification tree and want to prune it by using cost-complexity pruning in the rpart package. The objective function of cost-complexity prunig is C(T)=L(T)+a|T|. For regression tree ...
0
votes
0answers
9 views
Multi-class machine learning classification on binary data - setting a class for unclassifiable
I have a large dataset of binary genomic data (i.e. mutation Y/N)
A proportion of the samples have been classified into clusters based on presence of key mutations.
I would like to use this ...
1
vote
1answer
63 views
How to determine important variables in decision tree
I have created decision tree model on Auto dataset.
tree.auto = tree(highmpg ~ .,df)
I have attached the plot and copying the summary.
...
1
vote
1answer
28 views
Which value of accuracy or balanced accuracy is enough?
I constructed a classification tree and want validate the out of sample performance. I read that the accuracy or the balanced accuracy must at least higher than the no information rate. By the no ...
0
votes
1answer
31 views
Ctree in R: how optimal is the optimal split point?
Hi I’m fairly new to using decion trees. I understand that to find the best split points, the ctree algorithm maximises a certain test statistic. I am interested to inspect the values of the test ...
0
votes
0answers
4 views
Adding probabilistic suffix trees
In the context of Variable Length Markov Chains, I am interested in fitting VLMCs to longitudinal measurements of a set of patients without pruning and then add the trees together. Then, I could prune ...
0
votes
0answers
13 views
How to infer results from tree-based feature selection and chi squared?
I have a data set with 12 continuous features, with 3 discrete output labels. I want to determine the two best features.
My research thus far has led me to use chi^2 tests and extra tree classifiers ...
0
votes
0answers
27 views
Is it useful to compute R Squared for regression trees? [duplicate]
I have a regression tree and want to validate the peformance. The first measure I have is the mse to find which model is the best. After that I want to check if the model peforms better then an ...
5
votes
1answer
95 views
Should OOB (Out Of Bag) error be less than a Test set error in Random Forests?
I am using the book, "An introduction to statistical learning with applications in R" and reading the section on using OOB to estimate the model error for Random Forests. The graph seems to suggest ...
0
votes
1answer
42 views
Can regression trees outperform classification trees?
I'm learning about trees in my data science class recently and I know that the convention is to use regression trees when the response is continuous and classification trees when the response is ...
0
votes
0answers
17 views
Variance-bias trade off in classification and regression trees
I know what the variance/bias trade off is when I am talking about regression problems. Also in this context i understand the technical derivation.
But I don't have an idea of the variance-bias trade ...
0
votes
0answers
24 views
Using decision tree + active learning for regression?
Existing literatures that concerns using decision tree to do regression is more limited compared to its classification companion. The same also holds for research regarding active learning. I am just ...
0
votes
1answer
46 views
Which algorithm does Decision Tree classifier in sklearn implement?
Which algorithm does Decision Tree classifier in sklearn Library implement? Is it GUIDE?
There are a total of 6 techniques available according to my knowledge, according to this paper
0
votes
0answers
23 views
What is the meaning of “extremity of values for regression” when using sklearn.tree.export_graphviz?
I'm using sklearn.ensemble.RandomForestRegressor, and I would like to show the decision tree of one estimator using sklearn.tree.export_graphviz, but I don't understand what the meaning of extremity ...
1
vote
1answer
58 views
Do decision tree's perform variable selection?
I'm a bit confused how decision tree's select the variables to split.
I know they splitt the data set through variable to get a more pure data set.
But can it happend that some explenatory variables ...
0
votes
0answers
18 views
Why does Weka output decision tree with multiple children nodes of the same target variable?
I'm working with this dataset. I broke the quality class into 3 categories: low, medium and <...
1
vote
0answers
32 views
Wrong calculation of feature importance of decision tree in R [closed]
I trained decision tree both in python and R, but I think the way feature importance is calculated in R may be wrong. Following is the sample code which you can use to reproduce the problem. Let's say ...
1
vote
2answers
36 views
Can a decision tree make a decision based on two variables at one split?
I know that the random forest algorithm works by generating a set of decision trees with a subset of features.
Say I was using random forest as a classification algorithm looking at someone's data ...
2
votes
0answers
18 views
How should I configure CART for easier interpretation?
My goal is not to get better accuracy, but to make interpretation easier and have more stable splits.
So far most useful is use larger min_samples_leaf, but I always get nodes size almost same as ...
2
votes
3answers
263 views
What are the advantages of Random Forest over Decision Trees
I am currently searching for the advantages of random forest over decision trees, but unfortunately I didn't find a research paper that does such a conclusion that summarize all the advantages of RF ...
1
vote
0answers
24 views
Why we get different important factors when performing logistic regression and decision tree?
Why we get different important factors when performing logistic regression and decision tree? When I select features, I perform logistic regression and decision tree separately and wanna see the ...
0
votes
0answers
25 views
Difference in model performance measures of train and test data sets
I am using CART classification technique by dividing a dataset into train and test sets. I have been using Mis-classification error, KS by rank ordering, AUC and Gini as MPMs(model performance ...
2
votes
1answer
35 views
Formal Definition of Tree-Consistent
I am working my way through this paper on Bayesian Hierarchical Clustering. I keep seeing the phrase tree-consistent. However, it doesn't seem to be defined anywhere in the paper. There is a ...
1
vote
0answers
16 views
How do I reconcile the output of a decision tree visual vs the output of the DecisionTreeClassifier class?
I have a decision tree model stored in dt. I also plotted and can visually see the decision tree itself. Given a new, unseen customer, I can use the visual tree to ...
0
votes
0answers
13 views
what does the DeltaCriterionDecisionSplit and NumPredictorSplit in CompactTreeBagger (Matlab)means?
when I try to use CompactTreeBagger in Matlab, I do not know what does the DeltaCriterionDecisionSplit and NumPredictorSplit means. Even though I search the offical guidence https://ww2.mathworks.cn/...
1
vote
0answers
26 views
What is the connection of correlation between variables to decision tree feature importance
I am using decision tree regression to predict a variable.
I realized that the highest correlating features to my target variable are not the ones I get returned with a high feature importance.
...
1
vote
1answer
58 views
Updating decision tree for new data
Lets say you have trained a decision tree for 40 gigs of data.
On Monday morning you receive 10Gig new data and produce some results quickly
to report to your boss.
Can you update the decision tree ...
0
votes
1answer
30 views
Can we predict the monthly sales amount of the coming month without knowing the values of the independent variables of the coming month
I have a data set where the monthly sales of TMT bars and various other explanatory variables are present from April 2014-March 2018. I need to predict the monthly sales of the coming/next month.
...
4
votes
2answers
188 views
Is it ever recommended to use mean/multiple imputation when using tree-based predictive models?
Everytime that I am making some predictive model and I have missing data I impute categorical variables with something like "UNKNOWN" and numerical variables with some absurd number that will never be ...
1
vote
0answers
30 views
Can CART models be used to select features for a logistic regression?
Can I use the features selected from the CART(Classification and Regression Trees) model and take those features and then model the logistic regression using those selected features? Then interpret ...
0
votes
1answer
27 views
How does the exhaustive search for training decision trees work?
I am reading Pattern Matching and Machine Learning by Bishop at the moment. The chapter about decision trees states that to train a decision tree, one can use a greedy algorithm. The book describes ...
2
votes
0answers
62 views
Accuracy and ROC for Logistic and Decision Tree
So I run a logistic regression and decision tree model using same data. The accuracy shows that the decision tree outperforms logistic slightly. However, my ROC curve shows that logistic is much ...
0
votes
0answers
6 views
Incorporating seasonality into Chaid models
I am trying to predict no. of policies sold by a particular insurance agents in a particular month. The problem with Insurance slaes is the sales peaks up in the jan,Feb, MArch month (due to tax ...
