'Classification And Regression Trees'. CART is a popular data mining technique.

learn more… | top users | synonyms (3)

0
votes
1answer
16 views

Explaining the numbers in a decision tree

Using the famous Iris data set with Julia decision tree classifier I get the following tree. ...
0
votes
0answers
9 views

Why is my classification tree predicting only a few classes but not every class?

My dependent variable is a categorical variable with 8 categories. I use rpart to fit a classification tree. The output tree's terminal nodes predict only categories 1, 3 and 7. It says nothing about ...
0
votes
0answers
11 views

How does tree algorithm choosing which node to split?

I know the idea is to choose the split that most improves the loss criterion. In the case I'm interested in, it is the square-error loss. Now how do you get to the condition $$ argmax_{1 \leq m \leq ...
0
votes
0answers
13 views

What are the main differences between regression trees and model trees?

I am currently carrying out research into decision trees for regression problems. In the literature for the M5 algorithm I have noticed that it mentions that "model trees" and "regression trees" are ...
0
votes
1answer
24 views

Feature selection step before decision tree?

I want to use rpart (a R package) to build a decision tree model. The data is a high-dimensional expression matrix, with ~50,000 predictors and ~500 samples. The response is a categorical variable. ...
0
votes
0answers
12 views

Saving and loading Regression Tree

Please, could you tell me how to save the tree below in a file and then load it in another session: Model = as.data.frame(aMatrix) train = sample (1: nrow(Model), nrow(Model)/2) tree.model ...
0
votes
0answers
3 views

Weights in adaboost/decision tree(cart)

I'm trying to implement adaboost using decision trees. But I'm confused over the weights. I am unable to understand how to incorporate weights in training process, how the formulas for node entropy ...
0
votes
0answers
6 views

Gini impurity and generalization error

Has anyone seen papers on relationship between information-based criterions (such as Gini impurity, information gain etc.) and generalization error? Is there theoretical justification of using such ...
0
votes
0answers
5 views

Gbm.plot y axis

I am fitting a boosted regression tree to count data. The response is distributed Poisson. When I plot the model's partial residuals using gbm.plot, the y-axis goes from -1 to 1. Are these plots ...
2
votes
1answer
31 views

In a CART model, why is the average of the leaf proportions equal to the total proportion only when the classes are unweighted?

Suppose I want to do binary classification (the two classes are 0 and 1) and I choose to work with a CART model. I first fit this model on a training set. (Note that I am using Python, and ...
0
votes
1answer
21 views

Do Random Forests use boosting

Ok so I think I have listened to a few wrong discussions on random forests because now I have a very confused question. With respect to Random Forests and bagging/bootstrapping, I'm good there. The ...
0
votes
0answers
4 views

Using Decision Rules to Make Cluster efect

I have a data set with 3 independent variables and 1 dependent variable. Dependent is play_golf Independents are Humidity, Pending_Chores, Wind I want to create "clusters" of rules and aggregate ...
0
votes
1answer
28 views

(Boosted) regression trees versus model trees - rule of thumb what to use when

I apply (boosted) regression trees to build predicitive models with continuous outcome (xgboost and gbm). While regression trees ...
0
votes
0answers
25 views

The formal proof of purity gain (information gain) formula for decision trees

Suppose we are constructing a binary decision tree, and are using gini impurity (purity gain) to choose the best feature for splitting a node. We also have only binary features and only two classes. ...
8
votes
1answer
327 views

Are tree estimators ALWAYS biased?

I'm doing a homework on Decision Trees, and one of the questions I have to answer is "Why are estimators built out of trees biased, and how does bagging help reduce their variance?". Now, I know that ...
1
vote
1answer
39 views

Handling missing continuous attribute values in ID3

I'm implementing the ID3 algorithm. I have an attribute which happens to be continuous like 12.21, 3.01, etc. AND have missing values which are marked as "NA". How I'm discretizing the data: I'm ...
0
votes
0answers
10 views

Counting parameters of a gradient boosted decision tree

Given the number of predictors and the depth of the trees, how many are the parameters of the models in a boosted decision tree? Is there a simple formula to count all the parameters of the model as ...
-1
votes
1answer
53 views

How does XGboost (Python) differentiate between a nominal variable and a continuous variable?

Assume the data in one dimension is (-1.0, 2.0, 2.5, 3.0, 5.0). Does XGboost regard it as a nominal or a continuous variable?
0
votes
0answers
13 views

correlation between attributes in fuzzy dataset

I am new in this field. I want to ask if it is any simple way how to measure correlation between two attributes in data set. Data are defined by fuzzy logic. I have own implementation of fuzzy ...
3
votes
0answers
19 views

Data reduction and xgboost(or other boosting and decisision tree methods)

I wonder, does data reduction(ex:factor analysis) have an impact on the result of boosting(ex:xgboost) or decision trees methods other than time gain?
1
vote
1answer
70 views

XGBoost (Extreme Gradient Boosting) or Elastic Net More Robust to Outliers

I have recently been doing work with predictive models for a continuous response. I am doing a comparison between Elastic Net (glmnet) package in R and XGBoost ...
3
votes
1answer
55 views

Decision Tree: Adding “important” feature doesn't necessarily improve prediction

I am using a decision tree to perform binary classification. I've found that a particular feature seems to be an important one; however, keeping it in my model doesn't yield better predictions (i.e. ...
0
votes
0answers
9 views

Incorporating prior class probability in decision trees

How do we incorporate prior class distributions in algorithms such as CART? I read that it would have an impact on the splitting of the tree (if the distribution is different than what we have in the ...
0
votes
0answers
11 views

How can you look at a tree and assess that it does not capture any interaction between the predictor variables?

Classification and Regression trees are very good at capturing interactions. How can you look at a tree and assess that it does not capture any interaction between the predictor variables?
1
vote
0answers
25 views

Decision tree completeness and unclassified data

I made a program that trains a decision tree built on the ID3 algorithm using an information gain function (Shanon entropy) for feature selection (split). Once I trained a decision tree I tested it to ...
0
votes
0answers
42 views

Predicting a complex response with regression trees, [duplicate]

I have a set of 8th order I-invariants which have been assigned three labels. I would like to predicting a complex response with regression trees using scikit Predicting a complex response with ...
1
vote
1answer
99 views

xgboost - what is the difference between the tree booster and the linear booster?

I am aware of gradient boosted trees. The extreme-gradient boosting algrithm is widely applied these days. What excactly is the difference between the tree booster (gbtree) and the linear booster ...
0
votes
0answers
51 views

cross validation vs bagging

I understand the ins and outs of the processes of both cross validation (partition the data set evenly, train on k-1 partitions, blah blah blah) and bagging (train M models composed of n observations ...
0
votes
0answers
25 views

Decision Trees: Why not this instead of Information Gain?

In Decision Trees one wants to say in which order one wants to put (splits on) Features in the tree. Say, for example we have two discretely valued Features F,G and the target Feature Y is binary ...
0
votes
0answers
14 views

How to take into account the cost of analysis (real moneyz)

Simple question: how to predefine the order of features in the decision tree? Complex question: I wonder how do you take into account the cost of the analysis? For example, we have 3 types of ...
0
votes
0answers
26 views

decision tree multiway vs binary splits

im not 100% sure but is their a definition or theorem for a decision tree an his splits. Each multiway split can be represented by a number of binary splits? For example a three-way split, can split ...
0
votes
0answers
29 views

How does R's C5.0 define a tree size?

When running a C5.0, e.g. with this script: ...
1
vote
1answer
61 views

Visualizing C5.0 Decision Tree?

Is there a direct way to visualize a c5.0 decision tree? Here my code: ...
1
vote
1answer
50 views

What parameter of GBM does gradient descent update after calculating gradient of loss function?

I am going through Elements of statistical Learning and trying to understand GBM algorithm. The algorithm of GBM is shown below. I understand general gradient descent algorithm mentioned below very ...
0
votes
0answers
28 views

variable selection before a decision tree

I want to built a predictive decision tree. I have a dataset with +/- 1000 observations and 1500 variables. Can I just built my decision tree (training + validation dataset) with all the 1500 ...
0
votes
1answer
57 views

Decision Tree with continuous input variable

It is known that when constructing a decision tree, we split the input variable exhaustively and find the 'best' split by statistical test approach or Impurity function approach. My question is when ...
1
vote
1answer
68 views

Using trees after variable selection using Lasso/Random

I am new into Machine Learning so please excuse me if my question is naive. My question is, is it possible to use trees for example rpart or ctree after variable selection procedures such as ...
3
votes
1answer
48 views

Duplicate Training Data in Decision Tree Learning Algortihm

I have the following training data set where the first line shows the name of attributes. ...
0
votes
0answers
13 views

Correct Terminology of Information Gain

I'm trying to correctly understand terminology of information gain, entropy and Gini impurity. Do I understand it correctly in this way? Entropy change and Gini impurity are both "just" metric of ...
1
vote
1answer
23 views

number of trees that were built without minority class?

Lets assume that my random forest has 500 trees. My data is imbalance with 90% of class A and 10% of class B. I am wonder if there is any way to calculate roughly the number of trees that are built ...
1
vote
1answer
89 views

Choosing alpha for cost complexity pruning as described in Introduction to Statistical Learning

In the following lectures Tree Methods, they describe a tree algorithm for cost complexity pruning on page 21. It says we apply cost complexity pruning to the large tree in order to obtain a sequence ...
1
vote
2answers
247 views

Is decision tree output a prediction or class probabilities?

A Random Forest works by aggregating the results of many decision trees. Recently, I was reading about how the RandomForest aggregates the results, and it made me question whether the results from ...
2
votes
2answers
41 views

Why is CHAID (decision tree) analysis used in direct marketing? What makes it more suitable than other types of trees?

According to wikipedia CHAID is popular for modeling reponses in direct marketing (and I have seen it come up several times in this context). Does anyone know what it is that makes it ...
0
votes
0answers
21 views

Interpret C5.0 rules

According C5.0 documentation (https://www.rulequest.com/see5-unix.html#CASEWEIGHT) sample weights should not be a part of model and they should be used only during training. The case weight ...
0
votes
0answers
65 views

R C5.0 tree model to list conversion

I am using the C5.0 decision tree in R from the C50 package. The training function C5.0 returns a list which also contains a "tree" element which is basically a text representation of the tree. I am ...
3
votes
0answers
26 views

Can a decision tree automatically detect the effect on the dependent variable from the product/quotient of two independent variables?

For example, when I use the xgboost algorithm, there are two continuous variables X1 and X2, do I need to specify the product X1*X2 explicitly at the beginning? Or the algorithm can automatically pick ...
4
votes
1answer
130 views

How does the complexity parameter correspond to the number of splits in cross validation in rpart?

library(rpart) tree = rpart(Kyphosis ~ ., data=kyphosis, control=rpart.control(minsplit = 1, cp = 0, xval=10)) plotcp(tree, minline = FALSE, upper=c("splits")) ...
2
votes
1answer
37 views

Boosted Trees: Objective Function clarification

Reading through this overview of boosted trees, I'm having trouble understanding how the second line was derived. $$ Obj(t)=\sum_1^n{loss(y_{i} - \hat{y}_i^{(t)})} + \sum_1^t{\Omega(f_i)} \\ = ...
0
votes
0answers
29 views

Analysis of wrapper feature selection ouptput in Weka

I am using Weka to select important features from a dataset. I am using the wrapper method in this application. I chose a decision tree (j.48) for my classifier and Genetic search for the search ...
3
votes
1answer
39 views

Clustering patients according to biomarkers: an easy way out?

I've just started reading about clustering and classification. It's a djungle, a fascinating one. Currently, however I have a rather urgent task, i.e to perform a sort of cluster analysis in the sense ...