'Classification And Regression Trees'. CART is a popular data mining technique.

learn more… | top users | synonyms (3)

0
votes
0answers
8 views

number of trees that were built without minority class?

Lets assume that my random forest has 500 trees. My data is imbalance with 90% of class A and 10% of class B. I am wonder if there is any way to calculate roughly the number of trees that are built ...
1
vote
0answers
13 views

Choosing alpha for cost complexity pruning as described in Introduction to Statistical Learning

In the following lectures Tree Methods, they describe a tree algorithm for cost complexity pruning on page 21. It says we apply cost complexity pruning to the large tree in order to obtain a sequence ...
1
vote
2answers
46 views

Is decision tree output a prediction or class probabilities?

A Random Forest works by aggregating the results of many decision trees. Recently, I was reading about how the RandomForest aggregates the results, and it made me question whether the results from ...
2
votes
1answer
21 views

Why is CHAID (decision tree) analysis used in direct marketing? What makes it more suitable than other types of trees?

According to wikipedia CHAID is popular for modeling reponses in direct marketing (and I have seen it come up several times in this context). Does anyone know what it is that makes it ...
0
votes
0answers
15 views

Interpret C5.0 rules

According C5.0 documentation (https://www.rulequest.com/see5-unix.html#CASEWEIGHT) sample weights should not be a part of model and they should be used only during training. The case weight ...
0
votes
0answers
10 views

J48 partykit access flat list [migrated]

I would like to access individual nodes in the flat list representation of the party object. That is, I would like to get a node, its split and kids and any other ...
0
votes
0answers
19 views

R C5.0 tree model to list conversion

I am using the C5.0 decision tree in R from the C50 package. The training function C5.0 returns a list which also contains a "tree" element which is basically a text representation of the tree. I am ...
2
votes
0answers
15 views

Can a decision tree automatically detect the effect on the dependent variable from the product/quotient of two independent variables?

For example, when I use the xgboost algorithm, there are two continuous variables X1 and X2, do I need to specify the product X1*X2 explicitly at the beginning? Or the algorithm can automatically pick ...
3
votes
1answer
60 views

How does the complexity parameter correspond to the number of splits in cross validation in rpart?

library(rpart) tree = rpart(Kyphosis ~ ., data=kyphosis, control=rpart.control(minsplit = 1, cp = 0, xval=10)) plotcp(tree, minline = FALSE, upper=c("splits")) ...
2
votes
1answer
29 views

Boosted Trees: Objective Function clarification

Reading through this overview of boosted trees, I'm having trouble understanding how the second line was derived. $$ Obj(t)=\sum_1^n{loss(y_{i} - \hat{y}_i^{(t)})} + \sum_1^t{\Omega(f_i)} \\ = ...
0
votes
0answers
20 views

Analysis of wrapper feature selection ouptput in Weka

I am using Weka to select important features from a dataset. I am using the wrapper method in this application. I chose a decision tree (j.48) for my classifier and Genetic search for the search ...
3
votes
1answer
35 views

Clustering patients according to biomarkers: an easy way out?

I've just started reading about clustering and classification. It's a djungle, a fascinating one. Currently, however I have a rather urgent task, i.e to perform a sort of cluster analysis in the sense ...
3
votes
1answer
59 views

Decision Trees and Regression - Can predicted values be outside range of training data?

When it comes to decision trees, can the predicted value lay outside of the range of the training data? For example, if the training data set range of the target variable is 0-100, when I generate my ...
0
votes
0answers
50 views

Lack of fit for decision trees

I am using the cp argument in the rpart function in R. I would like to understand exactly how lack-of-fit is calculated for decision trees. Please provide a simple example if possible. Thanks. ...
5
votes
1answer
47 views

Does Random Forest ever compare the splitting of one node to the slitting of a **different** node?

I thought I understood how a single decision tree is constructed as part of a Random Forest : The data is split recursively until some kind of stopping conditions are met. Each split is ...
3
votes
3answers
79 views

Why does a regression tree not split based on variance?

When choosing each split, recursively, in a regression tree, I understand that you want to measure the spread, in each side of the split, essentially. So, in some sources, including this one at 6 ...
0
votes
0answers
12 views

Number of Dimensions in a CHAID Decision Tree

I am thinking of running a CHAID decision tree to understand which factors play an important role towards a store being highly profitability (1) or low profitability (0). I have 500 such stores being ...
0
votes
0answers
18 views

Training a Classification Tree with a binary split on each feature exactly once in R

I want to predict a binary variable y from predictors x1,...,xp using a decision tree. However, I want to enforce that the decision tree only contain binary splits, and that each predictor is split on ...
1
vote
1answer
73 views

How to interpret when logistic regression is performing better than classification trees?

I used both techniques on a rather unbalance data set (90%/10%) with 2500 observations and about 20 features. Firstly, I compared the default models. In a second step, I weighed the misclassification ...
0
votes
1answer
27 views

How to report decision tree statistics?

I need to do a formal report with the results of a decision tree classifier developed in SPSS, but I don't know how. I know there are really well defined ways to report statistics such as mean and ...
1
vote
0answers
23 views

Why do classification tress tend to perform better on large data sets and logistic regression on smaller ones?

Is this simply because if we have a lot of data, there is a bigger chance that a more complicated decision boundary (boundaries) is required to separate the observations? Does someone know researches ...
1
vote
0answers
23 views

Can the probabilities of classification trees be compared with the ones from logistic regression?

When I use the same data set to estimate a tree and a logistic regression model. Is there a way of comparing the resulting probabilities? In my view, there is not. Because, for logistic regression, I ...
3
votes
1answer
44 views

Selection bias in trees

In Applied Predictive Modeling by Kuhn and Johnson the authors write: Finally, these trees suffer from selection bias: predictors with a higher number of distinct values are favored over more ...
0
votes
1answer
77 views

How to compare classification methods in terms of performance?

I'd like to compare logistic regression to classification trees. In a first step, I compared the theoretical framework of the two classifiers. In a second step, I compared the performance using a ...
0
votes
0answers
29 views

Contradicting decision nodes in decision tree

My balance S is a continous variable. when it is not less than 200000 then it automatically means it is greater than equals to 200000 while decision tree is taking balance S >=200000 as another ...
0
votes
0answers
18 views

Is minimised cross-validated error equivalent to maximised cross-validated accuracy?

I'm running a classification tree using the function rpart. Usually, I choose the tree that minimises the cross-validation error. I was wondering, since error-rate is normally defined as 1-Accuracy. ...
2
votes
0answers
17 views

Entropy measure for multiple node data splitting with post-normalization constraint

Let me introduce my problem with a simple example. Let's say that we have two different classes $C_0$ and $C_1$ and we have one node $S$ that has the following elements of each class: $S = ...
2
votes
1answer
69 views

Does a linear recombination of features affect random forest?

I'm aware that linear transformations of individual features do not affect Random Forest. But what if features were linearly recombined to construct a new feature? I do the following: I take each ...
16
votes
3answers
416 views

How are Random Forests not sensitive to outliers?

I've read in a few sources, including this one, that Random Forests are not sensitive to outliers (in the way that Logistic Regression and other ML methods are, for example). However, two pieces of ...
1
vote
0answers
22 views

Consistent way of calculating average tree size using repeated cross validation?

I'd like to examine the behaviour of my classifier concerning the tree size. That is, I'm using rpart to build and prune the tree. For the rest of my analysis I'm running repeated 10 Fold Cross ...
2
votes
1answer
54 views

How to tune the “depth” and “min_samples_leaf” of Random Forest with correlated data?

I am having trouble with the intuition for running several RF models. I have a few features (between 3 and 10) which should be correlated, since they measure things that are usually related. I ...
4
votes
0answers
50 views

Let's share the Christmas spirit with a Christmas decision tree [closed]

The tree can be a regression or classification tree. It just needs to be grown automatically (e.g., via CART or C4.5) on a Christmas-related (possibly synthetic) data set. The final plot needs to ...
3
votes
1answer
48 views

Theoretical Justifications for Random Forest

Is there any theoretical justifications for Random Forests in high dimensions? I notice the work "Uniform Convergence of Random Forests via Adaptive Concentration" which shows generalization of RF ...
0
votes
0answers
34 views

Variable importance for conditional inference trees in R

I built a multivariate regression tree using the party package in R. The depth of the tree (max. number of splits) is 13. For the first 3/4 splits the tree is ...
0
votes
1answer
72 views

How to compare the performance of two classification methods? (logistic regression and classification trees)

I'm struggling a little bit with comparing these two classification methods. Although I know it is often a bad idea to use stepwise logistic-regression, I still want to perform it and analyse the ...
0
votes
0answers
28 views

xgboost: get error for each iteration

in XGBoost, is there a way to programmatically get the training and evaluation error per iteration of training? ...
0
votes
0answers
21 views

Repeated predictor in a CHAID tree

I'm trying to explain a variable (a mobility profile) thanks to predictors (for example age, gender, driver license, etc.) in a group of people. I'm using the CHAID algorithm with R. In the decision ...
1
vote
1answer
30 views

How to handle non- linear predictors in Decision Trees

I have a data set of just 1800 rows. I am not sure about how to provide inputs for certain variables. For eg., I have a column called number of previous jobs, and the values lie between 1 and 10, with ...
1
vote
0answers
14 views

Treating Categorical Variables as Continuous for Random Forest / Adaboost

What's the correct way to deal with categorical variables in packages like sklearn's RF and xgboost? Is there any cons of treating the variables are continuous? E.g. encode class A as 1, class B as ...
0
votes
0answers
19 views

Data mining for non-independent Data, in general?

Do most statistical learning or data mining methods assume that the observations are independent from each other? For example, for classification trees and the CART algorithm, while economists (e.g., ...
2
votes
1answer
73 views

CART and Clustered Data?

Just wonder if there is any caveat if one fits regular regression trees to clustered data but ignores the clustered structure of the data. More generally, how bad it would be if we fit regression ...
0
votes
0answers
30 views

Classification and random forests in Python: predictions are the same regardless of predictors

I'm working with a small data set of 4 categorical predictors, one binary outcome, and ~90k observations. I've tried fitting a random forest classifier mimicking the iris example from ...
1
vote
1answer
42 views

Interpreting regression inference trees

I am working with the ctree function that is implemented in R in the party and partykit ...
0
votes
1answer
19 views

How to measure quality of a split for numeric values?

I have a big set of real numbers. Each number comes with a list of associated attributes (some of them are numeric, others are categorical). For example, to make it less abstract, I have income of ...
0
votes
1answer
32 views

How to build classification model towards some rare response classes?

I was asked to build a predictive classification model that can predict some types of response. I am interested in 6 classes, however, the total occurence of these 6 classes (out of almost half a ...
3
votes
1answer
24 views

How does the ID3 Algorithm handle branching?

I'm a Computer Science student trying to implement the ID3 Algorithm and some of it isn't clear to me. The way I understand it, the ID3 algorithm involves building a (decision) tree by selecting an ...
2
votes
1answer
92 views

Recommend a method for variable selection (other than classification tree or random forest)?

Just wonder if you could recommend a few methods (other than tree-based methods) to analyze a dataset in which n= 350 and p = 35. The goal is not so much about prediction, but to find/select ...
2
votes
1answer
71 views

Fully grown decision trees in random forests

Several sources suggest it's ok to fully grow the decision trees in a RF (e.g., Leo Breiman's article and Elements of Statistical Learning, p. 596). I don't understand the following. Suppose that ...
4
votes
4answers
247 views

Does increase in training set size help in increasing the accuracy perpetually or is there a saturation point?

I am using a boosted trees classifier which is giving better accuracy then all other linear classifier I tried. I have almost an unlimited training data at my disposal , I wanted to know if there is a ...
0
votes
1answer
43 views

Balanced accuracy for decisions trees with unbalanced data

I have a question concerning decision trees and unbalanced data. My dependent variable accounts for around 2% of the entire dataset and is binary (0 or 1). Here are the steps I follow: Note that I'm ...