'Classification And Regression Trees'. CART is a popular data mining technique.

learn more… | top users | synonyms (3)

0
votes
0answers
2 views

How are decision trees built when weights are specified?

I have a decent understanding of the criteria by which a decision tree splits variables (information gain). However, I would like to understand what happens when we specify a weights vector. How does ...
1
vote
1answer
13 views

Minimizing Single-Class Classification Error in R Trees

I'm running tree models in R to help define rules for predicting a binary outcome (0 or 1, of course). I understand mostly how to algorithm works, but I'm in a position where I don't really care about ...
2
votes
2answers
23 views

Incorporating features that are always 0 given the value of another feature into a decision tree?

If I'm building a decision tree model, what is the best way to incorporate features that are always 0 given the value of another feature? For example, imagine I'm predicting whether or not someone ...
0
votes
1answer
17 views

Reduce false positives with XGBoost

I'm dealing with a dataset that contains almost same number of positive and negative samples (there are around 55% of positive samples and 45% of negative samples). With XGBoost I'm managing to ...
0
votes
0answers
10 views

Improving decision tree plot quality using rpart module

I am working on implementing decision trees to classify a target variable using R .I have around 21 features with around 800,000 rows.I am using rpart package to plot the tree I get,I have tried a lot ...
0
votes
0answers
17 views

Doing cross-validation when diagnosing a classifier through learning curves

I have a theoretical question on the correct way to make learning curves to diagnose a classifier. To see a generic example of these curves one can refer to this (min 34 onward) lecture by Andrew Ng ...
0
votes
0answers
16 views

root finding in CART

Brief Question: I haven't seen repeated "root finding" in CART content online. When I use it I mean it in the sense of: the interior max or min of continuous functions lives at the root (or zero) of ...
0
votes
0answers
20 views

Multiclass classification using CART in R

Hi I am building model to predict a multiclass classification variable with outcomes as 1,-1,0. Below is the model and classification matrix. ...
0
votes
1answer
19 views

What does the percentage output represent in RWeka M5P (tree model) output?

I have been teaching myself how to use RWeka, specifically so that I may implement the M5P model. I have been able to use apply to my data, but do not understand what the percentage represents. For ...
0
votes
0answers
20 views

Log transform in time series

After taking the $\log(1+x)$ transformation on a time series, I am guessing which features should I use as predictors: $\text{mean}(\log(1+x))$ vs $\log(1+\text{mean}(x))$ $\text{std}(\log(1+x))$ vs ...
0
votes
0answers
11 views

Decision Tree / Random Forest with constraints

I would like to add constraints during the process of selection of features during tree generation : By Expert knowledge/physical constraints, some feature are hiearchically on top. If stateA > ...
1
vote
1answer
45 views

How is the root node of a decision tree determined?

Almost all the examples I have found stated how the decision tree's split is based on how much purity/information can be gained (ie: via entropy and information gain) for internal node. But is the ...
0
votes
0answers
23 views

Train/test split for a small dataset (classification tree/ random forest)

I'm trying to solve a classification problem using Classification Tree with a small amount of data available. Depending on the content of particular train/test subsets (80:20 proportion) the total ...
0
votes
0answers
17 views

Algorithm for simplifying a deep tree?

Suppose that I have a very deep tree, which is not simplified, i.e many terminal nodes can be regrouped. Does exist an algorithm for simplifying this complicated tree? (The final tree is not ...
3
votes
2answers
84 views

Which algorithm can learn exactly a tree structure without noise?

Let's say I have a response variable $Y$ satisfying a complicated tree-structure: $$Y=f(X_1,X_2,X_3,\ldots,X_p) + \varepsilon$$ where: $\varepsilon = 0$ $f$ is a known deep tree-structure ...
0
votes
0answers
25 views

Rpart speed : number of variables more important than number of splits to try

I already posted this question on stack overflow, but since I didn't get any answer, I thought I may have more success here, not sure if this is allowed if not then sorry. I thought that the number ...
0
votes
0answers
5 views

Building a Regression Tree with one Gaussian Mixture Model at each node

I am trying to build a regression tree that outputs both a mean and a covariance matrix for each leaf of the tree. Ideally I would be able to have a Gaussian Mixture Model at each leaf. A first ...
1
vote
0answers
10 views

Variable Importance's reliability when model accuracy is low

I am trying to predict the termination events for a company. I also want to rank the most important variables that lead to termination. I ran a decision tree model and have the following questions: ...
1
vote
0answers
14 views

How many folds are used, cross validation with rpart printcp

I am using rpart to generate a decision tree and the printcp function (from the rpart package) to cross-validate. I have read the documentation and help pages, however I cannot find how many folds ...
0
votes
0answers
5 views

How does Quest differ from, for example, C&RT or C5.0?

SPSS Modeler has implementations of a number of decision tree data mining algorithms. Some of them are relatively well-known, such as C&RT and C5.0, some slightly less so, such as CHAID and QUEST. ...
0
votes
0answers
15 views

Difference between One Rule and Learn One Rule

What are the differences between One Rule and Learn One Rule? Learn One Rule goes deeper in the decision tree (and, like 1R, tests on all the attributes), is this correct? Thanks!
0
votes
1answer
37 views

Regression trees - how are splits decided

1) How do we decide if we split into 2 subnodes, or more? Or is it always 2? 2) How do we decide what threshold is the cutoff? Specifically, you have a continuous variable, do you do a binary search, ...
-1
votes
0answers
16 views

Deciding Minbucket size in R

What is the criteria for selecting minbucket size in CART in R? Is there any formula for choosing this, based on the number of observations?
0
votes
0answers
16 views

Adding variables to rpart stops the algorithm from splitting (returning base node)

I'm trying to figure out how exactly rpart determines the optimal split. I've read the documentation which states that it optimises for maximal reduction in impurity (information or gini), but it ...
0
votes
0answers
40 views

Plot a subtree from a big decision tree

I am working on my thesis using decision trees. I am presenting the resulting tree to show how they help in exploring data. My issue is that since the tree is big, I want to break it down into parts, ...
0
votes
0answers
15 views

Building a hierarchical classifier

I'm trying to build a hierarchical classifier on R. I know my data has a tree structure with a root node + 3 levels and that each node on a level has 10 nodes coming out of it. ...
0
votes
0answers
20 views

MCMC sampling of tree structure

I want to do Metropolis Hastings sampling of a tree structure $\mathcal{T}$ (in the space where the tree size is fixed) from a posterior distribution: $p(\mathcal{T}|X) \propto p(X|\mathcal{T})p(\...
0
votes
0answers
9 views

How to calculate entropy for a specific attribute?

This is super simple but I'm learning about decision trees and the ID3 algorithm. I found a website that's very helpful and I was following everything about entropy and information gain until I got ...
0
votes
1answer
27 views

Classification Problem with Big Unbalanced Data

I have a dataset with over 1 million observations, and a few dozens of predictors. My target variable is binary (gold customer vs. not gold customer) and I wish to build a classifier for prediction. ...
1
vote
0answers
12 views

Understanding Plurality Voting in Ensemble Methods

I'm reading Data mining with decision trees by Rokach, and i've got to a chapter about ensemble methods (using multiple classifiers) and this is where I can't wrap my head around this concept of ...
0
votes
1answer
24 views

Explanation of different testtype and teststats in ctree in party package of R

I was looking into the ctree function in the party package for R. In the ctree_control ...
0
votes
1answer
46 views

preprocessing and input format for gradient boosted trees

I'm using MlLib library de Spark and trying to perform Gradient-Boosted Trees algorithm on my data, that has mostly categorical features (and just two numerical features). in the example given in ...
0
votes
0answers
139 views

XGBoost - Can we find a “better” objective function than RMSE for regression?

If we think back to linear models for a moment, we have Ordinary Least Squares (OLS) versus Generalized Linear Models (GLM). Without going too in-depth, it can be said that GLMs "improve" upon OLS by ...
1
vote
0answers
23 views

Comparison of decision trees in R cran

Is it possible to compare two (or more) models of decision tree obtained with ctree on different data? In particular I would like a statistical test that compares two decision trees (builded with the ...
1
vote
1answer
26 views

Classification and Regression Trees

I would like some insight and some typical "rules of thumb" I can use to decide when to transform continuous predictors into factors and vice versa for classification and regression trees. Should I ...
0
votes
0answers
14 views

gradient boosted cart models - paradigm

Disclaimers: This is a question about the model, not the software. While I was exposed to boosting in my undergraduate program (2007) I have been using boosted trees on and off for the last ~4 years. ...
0
votes
0answers
18 views

Decision Tree with High Specificity and Low Sensitivity

I ran a decision tree on some data with over 1 million of observations, a binary dependent variable and a few dozens of independent variables, some partially correlated. I got weird results. The ...
0
votes
0answers
20 views

When do you use random forest over decision tree?

What criteria make you decide to use random forest over decision tree ? How do we decide when decision tree is not sufficient ?
0
votes
0answers
34 views

Are Logistic Model Trees robust to imbalanced datasets?

I am trying out Logistic Model Trees (Weka implementation) with very imbalanced datasets. Is anyone aware of their robustness without the need of preprocessing steps (such as SMOTE)? My preliminary ...
0
votes
1answer
64 views

Performance of regresion tree rpart

I am running a regression tree using rpart and I would like to understand how well it is performing. I know that rpart has cross validation built in, so I should not divide the dataset before of the ...
0
votes
1answer
24 views

Decision Tree Giving Impossible Splits

I ran a decision tree in SPSS, using the CHAID method. The result was a tree with many nodes. Some of the splits were impossible. For example: for a variable that is from 0 to 10 (in %), a split was >...
0
votes
0answers
9 views

Rpart maximum depth

Right now I am using the Rpart library to classify text sentiment, but a problem that I have run into is that the maximum depth of the tree is 30. As a result when I use more than 400 features (or in ...
0
votes
0answers
9 views

Why is $\Delta r$ equal to 0?

From the Rpart documentation it gives an example as to why $\Delta r$, the change in risk is a bad indicator to use to determine whether the tree should be split, here is the paragraph: My question ...
0
votes
0answers
42 views

Scaling the data in a decision tree changed my results?

I know that a decision tree doesn't get affected by scaling the data but when I scale the data within my decision tree it gives me a bad performance (bad recall, precision and accuracy) But when I ...
0
votes
0answers
15 views

Predictive power CART pruned vs. unpruned tree

I have a large sales dataset and was intending to use a CART tree to predict the sales price of each item depending on lots of input factors such as the sales region etc. To achieve this I used the ...
2
votes
3answers
149 views

Categorizing Continuous Random Variable in Logistic Regression

I have a Bernoulli response variable and I am going to fit a logistic regression. One of my independent variables is a continuous random variable and I would like to categorize it before fitting the ...
0
votes
0answers
15 views

Ordered sets vs randomly sorted sets: if the subset c(i,i,i,i) is random, is it possible to make the same prediction for the whole subset?

The "compared", "random", and "day" variables are categorical. Here is the sorted data: ...
0
votes
0answers
51 views

Can a classification tree model “know” to predict only one of every class for every subset in a data?

In order to help recipients understand my question, there will be context added. I don't know a whole lot of semantics so please bare with me. Draper is hosting a competition on Kaggle to classify ...
0
votes
1answer
55 views

Text & Data Mining Techniques for Twitter data

I have been using Statistica tool to do text mining analysis for Twitter data. Can any one tell me what kind of analysis (Text & Data mining) we can do with Twitter data in general. I am very much ...
0
votes
0answers
12 views

How to train decision tree with adaboost?

I am trying to implement decision tree with adaboost. I understand how the normal decision tree works with the entropy formula. I also understand how adaboost works. But i dont understand how adaboost ...