Questions tagged [cart]

'Classification And Regression Trees'. CART is a popular data mining technique.

937 questions
12 views

How the probability in a decision tree is calculated? [closed]

I build a classification tree and predicted the probability that an observation belong to the positive class. Now i ask myself how this probability is calculated?
14 views

Decision Trees - Regression trees weighting of child nodes?

I'm familiar with how classification trees weight the impurity measure of a potential split by the proportion of observations that would fall into each child node, such as:  loss = \frac{n_1}{N_m} ...
38 views

Gini Index Formula

I've read many related articles and posts. The more I read, the more I got confused about 'Gini index' and 'Gini Impurity'. I understood the concept but it seems to me that these things are used ...
27 views

Gradient Boosting regression model produces inaccurate forecasts in presence of outliers in a training set [closed]

I'm using XGBoost with a 'reg:linear' objective in order to forecast time series one step forward. I notice that the model appear to start producing forecasts that are too high when there were high-...
16 views

Decision tree when leaf nodes are numerical values

I'm having a dataset, where I created a decision tree. My output variables are binary values. I selected two features and the decision tree was generated using R. This is my code. ...
15 views

How to deal with repeated words in training and testing set using CART?

I have a dataset where a child read a passage out loud. Each row is one word (I’m order of the passage). I have a 1/0 for whether they read the word incorrectly or correctly. I’m trying to predict ...
3 views

Relationship between CART single node tree, variable correlation and variable variance

I am currently working on finding a pattern of when CART produces single node trees after cost-complexity pruning. I am most interested in the effect of variable variance and variable correlation. ...
30 views

Are feature importances from tree based models directly actionable for business?

If my response variable say is "has_repurchased" [0 or 1] and I have all customer level features. Can I rank the features in order of importance from the random forest model and report them as whats ...
55 views

Can balanced accuracy be higher than accuracy?

I have classification tree where the balanced accuracy of the test set is higher than the normal accuracy. I thought balanced accuracy can only have at his maximum the same value as the accuracy not ...
57 views

How to check if i have strong linear relationship between dependent variable and independent variables in linear regression (OLS)?

I want compare the out of sample prediction from an linear regression model (OLS) and a regression tree. I read that OLS outperforms regression tree if the relationship between the dependent variable ...
13 views

Calculating a continuous variable with regression trees

I have sample records with several attributes (predictors) and a predicted variable Yes/No. What I need is, given new data that omits the column Yes/No, to know what is the probability of Yes. Note ...
27 views

regression tree vs linear regression

I'm using one explanatory variable in a regression tree and in a linear regression. The tree finds a split (with variance reduction splitting rule), though R2 is pretty small (0.2). On the validation ...
2k views

Is random forest for regression a 'true' regression?

Random forests are used for regression. However, from what I understand, they assign an average target value at each leaf. Since there are only limited leaves in each tree, there are only specific ...
56 views

Performance Imbalance Dataset Decision Tree

I have a imbalance dataset for a classification task, with the minority class accounting for about 21% of the total. When I use a decision tree based model for prediction, let's say a classification ...
75 views

How regression trees split, when all the Features and target have only continuous values

Can anyone please explain how splitting is performed in regression trees when we only have continuous features. I have referred to different papers, but all I could find is formulas or theorems. Can ...
9 views

conditional trees unexpected grouping — differences with chi squared test

I grouped some data with ctree (party package, conditional trees) (binary target) and the result is: ...
40 views

R result interpretation conditional inference tree result for nominal response

I have nominal responses, "yes/no/don't know", that I am using in a conditional inference tree in R. I am having trouble with how to interpret the model's output concerning one of the independent ...
23 views

What is the objective function in cost-complexity pruning in rpart for classification tree?

I constructed a classification tree and want to prune it by using cost-complexity pruning in the rpart package. The objective function of cost-complexity prunig is C(T)=L(T)+a|T|. For regression tree ...
9 views

Multi-class machine learning classification on binary data - setting a class for unclassifiable

I have a large dataset of binary genomic data (i.e. mutation Y/N) A proportion of the samples have been classified into clusters based on presence of key mutations. I would like to use this ...
63 views

How to determine important variables in decision tree

I have created decision tree model on Auto dataset. tree.auto = tree(highmpg ~ .,df) I have attached the plot and copying the summary. ...
28 views

Which value of accuracy or balanced accuracy is enough?

I constructed a classification tree and want validate the out of sample performance. I read that the accuracy or the balanced accuracy must at least higher than the no information rate. By the no ...
31 views

Ctree in R: how optimal is the optimal split point?

Hi I’m fairly new to using decion trees. I understand that to find the best split points, the ctree algorithm maximises a certain test statistic. I am interested to inspect the values of the test ...
4 views

In the context of Variable Length Markov Chains, I am interested in fitting VLMCs to longitudinal measurements of a set of patients without pruning and then add the trees together. Then, I could prune ...
13 views

How to infer results from tree-based feature selection and chi squared?

I have a data set with 12 continuous features, with 3 discrete output labels. I want to determine the two best features. My research thus far has led me to use chi^2 tests and extra tree classifiers ...
27 views

Is it useful to compute R Squared for regression trees? [duplicate]

I have a regression tree and want to validate the peformance. The first measure I have is the mse to find which model is the best. After that I want to check if the model peforms better then an ...
95 views

Should OOB (Out Of Bag) error be less than a Test set error in Random Forests?

I am using the book, "An introduction to statistical learning with applications in R" and reading the section on using OOB to estimate the model error for Random Forests. The graph seems to suggest ...
42 views

Can regression trees outperform classification trees?

I'm learning about trees in my data science class recently and I know that the convention is to use regression trees when the response is continuous and classification trees when the response is ...
17 views

Variance-bias trade off in classification and regression trees

I know what the variance/bias trade off is when I am talking about regression problems. Also in this context i understand the technical derivation. But I don't have an idea of the variance-bias trade ...
24 views

Using decision tree + active learning for regression?

Existing literatures that concerns using decision tree to do regression is more limited compared to its classification companion. The same also holds for research regarding active learning. I am just ...
46 views

Which algorithm does Decision Tree classifier in sklearn implement?

Which algorithm does Decision Tree classifier in sklearn Library implement? Is it GUIDE? There are a total of 6 techniques available according to my knowledge, according to this paper
23 views

What is the meaning of “extremity of values for regression” when using sklearn.tree.export_graphviz?

I'm using sklearn.ensemble.RandomForestRegressor, and I would like to show the decision tree of one estimator using sklearn.tree.export_graphviz, but I don't understand what the meaning of extremity ...
58 views

Do decision tree's perform variable selection?

I'm a bit confused how decision tree's select the variables to split. I know they splitt the data set through variable to get a more pure data set. But can it happend that some explenatory variables ...
18 views

Why does Weka output decision tree with multiple children nodes of the same target variable?

I'm working with this dataset. I broke the quality class into 3 categories: low, medium and <...
32 views

Wrong calculation of feature importance of decision tree in R [closed]

I trained decision tree both in python and R, but I think the way feature importance is calculated in R may be wrong. Following is the sample code which you can use to reproduce the problem. Let's say ...
36 views

Can a decision tree make a decision based on two variables at one split?

I know that the random forest algorithm works by generating a set of decision trees with a subset of features. Say I was using random forest as a classification algorithm looking at someone's data ...
18 views

How should I configure CART for easier interpretation?

My goal is not to get better accuracy, but to make interpretation easier and have more stable splits. So far most useful is use larger min_samples_leaf, but I always get nodes size almost same as ...
263 views

What are the advantages of Random Forest over Decision Trees

I am currently searching for the advantages of random forest over decision trees, but unfortunately I didn't find a research paper that does such a conclusion that summarize all the advantages of RF ...
24 views

Why we get different important factors when performing logistic regression and decision tree?

Why we get different important factors when performing logistic regression and decision tree? When I select features, I perform logistic regression and decision tree separately and wanna see the ...
25 views

Difference in model performance measures of train and test data sets

I am using CART classification technique by dividing a dataset into train and test sets. I have been using Mis-classification error, KS by rank ordering, AUC and Gini as MPMs(model performance ...
35 views

Formal Definition of Tree-Consistent

I am working my way through this paper on Bayesian Hierarchical Clustering. I keep seeing the phrase tree-consistent. However, it doesn't seem to be defined anywhere in the paper. There is a ...
16 views

How do I reconcile the output of a decision tree visual vs the output of the DecisionTreeClassifier class?

I have a decision tree model stored in dt. I also plotted and can visually see the decision tree itself. Given a new, unseen customer, I can use the visual tree to ...
13 views

what does the DeltaCriterionDecisionSplit and NumPredictorSplit in CompactTreeBagger (Matlab)means?

when I try to use CompactTreeBagger in Matlab, I do not know what does the DeltaCriterionDecisionSplit and NumPredictorSplit means. Even though I search the offical guidence https://ww2.mathworks.cn/...
26 views

What is the connection of correlation between variables to decision tree feature importance

I am using decision tree regression to predict a variable. I realized that the highest correlating features to my target variable are not the ones I get returned with a high feature importance. ...
58 views

Updating decision tree for new data

Lets say you have trained a decision tree for 40 gigs of data. On Monday morning you receive 10Gig new data and produce some results quickly to report to your boss. Can you update the decision tree ...
30 views

Can we predict the monthly sales amount of the coming month without knowing the values of the independent variables of the coming month

I have a data set where the monthly sales of TMT bars and various other explanatory variables are present from April 2014-March 2018. I need to predict the monthly sales of the coming/next month. ...
188 views

Is it ever recommended to use mean/multiple imputation when using tree-based predictive models?

Everytime that I am making some predictive model and I have missing data I impute categorical variables with something like "UNKNOWN" and numerical variables with some absurd number that will never be ...
30 views

Can CART models be used to select features for a logistic regression?

Can I use the features selected from the CART(Classification and Regression Trees) model and take those features and then model the logistic regression using those selected features? Then interpret ...
27 views

How does the exhaustive search for training decision trees work?

I am reading Pattern Matching and Machine Learning by Bishop at the moment. The chapter about decision trees states that to train a decision tree, one can use a greedy algorithm. The book describes ...