'Classification And Regression Trees'. CART is a popular data mining technique.

learn more… | top users | synonyms (3)

0
votes
0answers
14 views

Decision Tree Giving Impossible Splits

I ran a decision tree in SPSS, using the CHAID method. The result was a tree with many nodes. Some of the splits were impossible. For example: for a variable that is from 0 to 10 (in %), a split was ...
0
votes
0answers
6 views

Rpart maximum depth

Right now I am using the Rpart library to classify text sentiment, but a problem that I have run into is that the maximum depth of the tree is 30. As a result when I use more than 400 features (or in ...
0
votes
0answers
7 views

Set minimum Leaf size for ctree [on hold]

Is there a method to set the minimum number of samples (leaf size) using ctree (library party) in R cran?
0
votes
0answers
9 views

Why is $\Delta r$ equal to 0?

From the Rpart documentation it gives an example as to why $\Delta r$, the change in risk is a bad indicator to use to determine whether the tree should be split, here is the paragraph: My question ...
0
votes
0answers
32 views

Scaling the data in a decision tree changed my results?

I know that a decision tree doesn't get affected by scaling the data but when I scale the data within my decision tree it gives me a bad performance (bad recall, precision and accuracy) But when I ...
0
votes
0answers
12 views

Predictive power CART pruned vs. unpruned tree

I have a large sales dataset and was intending to use a CART tree to predict the sales price of each item depending on lots of input factors such as the sales region etc. To achieve this I used the ...
2
votes
1answer
94 views
+50

Categorizing Continuous Random Variable in Logistic Regression

I have a Bernoulli response variable and I am going to fit a logistic regression. One of my independent variables is a continuous random variable and I would like to categorize it before fitting the ...
0
votes
0answers
14 views

Ordered sets vs randomly sorted sets: if the subset c(i,i,i,i) is random, is it possible to make the same prediction for the whole subset?

The "compared", "random", and "day" variables are categorical. Here is the sorted data: ...
0
votes
0answers
47 views

Can a classification tree model “know” to predict only one of every class for every subset in a data?

In order to help recipients understand my question, there will be context added. I don't know a whole lot of semantics so please bare with me. Draper is hosting a competition on Kaggle to classify ...
0
votes
0answers
13 views

How do I convert an “RWeka” decision tree into a “party” tree in R? [migrated]

I am using the RWeka package in R to fit M5' trees to a dataset using "M5P". I then want to convert the tree generated into a "party" tree so that I can access variable importances. The issue I am ...
0
votes
2answers
24 views

Text & Data Mining Techniques for Twitter data

I have been using Statistica tool to do text mining analysis for Twitter data. Can any one tell me what kind of analysis (Text & Data mining) we can do with Twitter data in general. I am very much ...
0
votes
0answers
9 views

How to train decision tree with adaboost?

I am trying to implement decision tree with adaboost. I understand how the normal decision tree works with the entropy formula. I also understand how adaboost works. But i dont understand how adaboost ...
1
vote
1answer
33 views

Relation between calculated probabilites and Decision Tree

I'm currently building a classification model in MS azure with the Two-Class Boosted Decision Tree algorithm. From my basic knowledge I know that the decision tree splits the features by a cut value ...
0
votes
3answers
40 views

Multiclass classification question

I am working on applying Random Forests to a multiclass classification problem, where I have a set of 11 predictor variables and a response that can take the values of "Yes", "No", and "Maybe". In my ...
1
vote
0answers
11 views

How to mitigate the hierarchical error propagation in tree-structured classification

Suppose we have a multi-class classification problem, where the number of classes $K \geq 3$ We use a tree structure of multiple SVMs to divide and conquer the problem, with one example in the figure ...
1
vote
2answers
27 views

Boosted Trees classification

I'm using R's gbm() package to do a boosted classification problem, where my response variable is a binary variable taking values of 0 and 1. I have 11 predictors in my data set. After running the ...
0
votes
1answer
24 views

Explaining the numbers in a decision tree

Using the famous Iris data set with Julia decision tree classifier I get the following tree. ...
0
votes
0answers
11 views

Why is my classification tree predicting only a few classes but not every class?

My dependent variable is a categorical variable with 8 categories. I use rpart to fit a classification tree. The output tree's terminal nodes predict only categories 1, 3 and 7. It says nothing about ...
0
votes
0answers
16 views

How does tree algorithm choosing which node to split?

I know the idea is to choose the split that most improves the loss criterion. In the case I'm interested in, it is the square-error loss. Now how do you get to the condition $$ argmax_{1 \leq m \leq ...
0
votes
0answers
16 views

What are the main differences between regression trees and model trees?

I am currently carrying out research into decision trees for regression problems. In the literature for the M5 algorithm I have noticed that it mentions that "model trees" and "regression trees" are ...
0
votes
1answer
28 views

Feature selection step before decision tree?

I want to use rpart (a R package) to build a decision tree model. The data is a high-dimensional expression matrix, with ~50,000 predictors and ~500 samples. The response is a categorical variable. ...
0
votes
0answers
12 views

Saving and loading Regression Tree

Please, could you tell me how to save the tree below in a file and then load it in another session: Model = as.data.frame(aMatrix) train = sample (1: nrow(Model), nrow(Model)/2) tree.model ...
0
votes
0answers
5 views

Weights in adaboost/decision tree(cart)

I'm trying to implement adaboost using decision trees. But I'm confused over the weights. I am unable to understand how to incorporate weights in training process, how the formulas for node entropy ...
0
votes
0answers
7 views

Gini impurity and generalization error

Has anyone seen papers on relationship between information-based criterions (such as Gini impurity, information gain etc.) and generalization error? Is there theoretical justification of using such ...
0
votes
0answers
7 views

Gbm.plot y axis

I am fitting a boosted regression tree to count data. The response is distributed Poisson. When I plot the model's partial residuals using gbm.plot, the y-axis goes from -1 to 1. Are these plots ...
2
votes
1answer
35 views

In a CART model, why is the average of the leaf proportions equal to the total proportion only when the classes are unweighted?

Suppose I want to do binary classification (the two classes are 0 and 1) and I choose to work with a CART model. I first fit this model on a training set. (Note that I am using Python, and ...
0
votes
1answer
24 views

Do Random Forests use boosting

Ok so I think I have listened to a few wrong discussions on random forests because now I have a very confused question. With respect to Random Forests and bagging/bootstrapping, I'm good there. The ...
0
votes
0answers
6 views

Using Decision Rules to Make Cluster efect

I have a data set with 3 independent variables and 1 dependent variable. Dependent is play_golf Independents are Humidity, Pending_Chores, Wind I want to create "clusters" of rules and aggregate ...
0
votes
1answer
34 views

(Boosted) regression trees versus model trees - rule of thumb what to use when

I apply (boosted) regression trees to build predicitive models with continuous outcome (xgboost and gbm). While regression trees ...
0
votes
0answers
30 views

The formal proof of purity gain (information gain) formula for decision trees

Suppose we are constructing a binary decision tree, and are using gini impurity (purity gain) to choose the best feature for splitting a node. We also have only binary features and only two classes. ...
8
votes
1answer
330 views

Are tree estimators ALWAYS biased?

I'm doing a homework on Decision Trees, and one of the questions I have to answer is "Why are estimators built out of trees biased, and how does bagging help reduce their variance?". Now, I know that ...
1
vote
1answer
59 views

Handling missing continuous attribute values in ID3

I'm implementing the ID3 algorithm. I have an attribute which happens to be continuous like 12.21, 3.01, etc. AND have missing values which are marked as "NA". How I'm discretizing the data: I'm ...
0
votes
0answers
10 views

Counting parameters of a gradient boosted decision tree

Given the number of predictors and the depth of the trees, how many are the parameters of the models in a boosted decision tree? Is there a simple formula to count all the parameters of the model as ...
-1
votes
1answer
90 views

How does XGboost (Python) differentiate between a nominal variable and a continuous variable?

Assume the data in one dimension is (-1.0, 2.0, 2.5, 3.0, 5.0). Does XGboost regard it as a nominal or a continuous variable?
0
votes
0answers
29 views

correlation between attributes in fuzzy dataset

I am new in this field. I want to ask if it is any simple way how to measure correlation between two attributes in data set. Data are defined by fuzzy logic. I have own implementation of fuzzy ...
3
votes
0answers
21 views

Data reduction and xgboost(or other boosting and decisision tree methods)

I wonder, does data reduction(ex:factor analysis) have an impact on the result of boosting(ex:xgboost) or decision trees methods other than time gain?
1
vote
1answer
105 views

XGBoost (Extreme Gradient Boosting) or Elastic Net More Robust to Outliers

I have recently been doing work with predictive models for a continuous response. I am doing a comparison between Elastic Net (glmnet) package in R and XGBoost ...
3
votes
1answer
60 views

Decision Tree: Adding “important” feature doesn't necessarily improve prediction

I am using a decision tree to perform binary classification. I've found that a particular feature seems to be an important one; however, keeping it in my model doesn't yield better predictions (i.e. ...
0
votes
0answers
10 views

Incorporating prior class probability in decision trees

How do we incorporate prior class distributions in algorithms such as CART? I read that it would have an impact on the splitting of the tree (if the distribution is different than what we have in the ...
0
votes
0answers
12 views

How can you look at a tree and assess that it does not capture any interaction between the predictor variables?

Classification and Regression trees are very good at capturing interactions. How can you look at a tree and assess that it does not capture any interaction between the predictor variables?
1
vote
0answers
29 views

Decision tree completeness and unclassified data

I made a program that trains a decision tree built on the ID3 algorithm using an information gain function (Shanon entropy) for feature selection (split). Once I trained a decision tree I tested it to ...
0
votes
0answers
43 views

Predicting a complex response with regression trees, [duplicate]

I have a set of 8th order I-invariants which have been assigned three labels. I would like to predicting a complex response with regression trees using scikit Predicting a complex response with ...
1
vote
1answer
151 views

xgboost - what is the difference between the tree booster and the linear booster?

I am aware of gradient boosted trees. The extreme-gradient boosting algrithm is widely applied these days. What excactly is the difference between the tree booster (gbtree) and the linear booster ...
0
votes
0answers
62 views

cross validation vs bagging

I understand the ins and outs of the processes of both cross validation (partition the data set evenly, train on k-1 partitions, blah blah blah) and bagging (train M models composed of n observations ...
0
votes
0answers
26 views

Decision Trees: Why not this instead of Information Gain?

In Decision Trees one wants to say in which order one wants to put (splits on) Features in the tree. Say, for example we have two discretely valued Features F,G and the target Feature Y is binary ...
0
votes
0answers
15 views

How to take into account the cost of analysis (real moneyz)

Simple question: how to predefine the order of features in the decision tree? Complex question: I wonder how do you take into account the cost of the analysis? For example, we have 3 types of ...
0
votes
0answers
28 views

decision tree multiway vs binary splits

im not 100% sure but is their a definition or theorem for a decision tree an his splits. Each multiway split can be represented by a number of binary splits? For example a three-way split, can split ...
0
votes
0answers
39 views

How does R's C5.0 define a tree size?

When running a C5.0, e.g. with this script: ...
1
vote
1answer
110 views

Visualizing C5.0 Decision Tree?

Is there a direct way to visualize a c5.0 decision tree? Here my code: ...
1
vote
1answer
58 views

What parameter of GBM does gradient descent update after calculating gradient of loss function?

I am going through Elements of statistical Learning and trying to understand GBM algorithm. The algorithm of GBM is shown below. I understand general gradient descent algorithm mentioned below very ...