Tagged Questions

Stands for 'Classification And Regression Trees'. CART is a technique for developing a tree model (T) to predict categories (C) and/or continuous values (R) by recursive partitioning. It does not make restrictive parametric assumptions. CART is a popular data mining technique.

learn more… | top users | synonyms (2)

1
vote
2answers
80 views

How can I get more precise regression tree?

I am a complete newbie to regression trees so maybe I am not understanding it properly. I got the following tree from my analysis (function tree() from R package ...
1
vote
0answers
12 views

Interpreting output from cvFit(), understanding cross-validation in classification tree model

I am trying to understand how to interpret the output for cvFit(). The data is from UCI's ML repository. This is my model ...
0
votes
0answers
8 views

Scoring outputs from multiple decision trees

What is the best way to score outputs of multiple decision trees that classify instances of different classes? Say I have two decision trees. One tries to classify instances to class A, another one ...
0
votes
0answers
11 views

Degrees of freedom when splitting populations

I have a question relating to degrees of freedom when I have a number parameters in a model for a population, and then I want to split the population. Say for a certain population I have a the ...
0
votes
1answer
16 views

Suitable function to choose the best split in a regression tree/oblivious tree

My main objective is to construct a regression (decision) tree. It is a part of a boosting algorithm using additive regression trees. The first question is what other functions (other than least ...
0
votes
0answers
12 views

tree - recursive partitioning

I am trying to understand recursive partitioning. I am simulating data. I am curious as to why, subsetting on var2, the coefficient estimates for var1 and var3 are becoming similar to those of the ...
0
votes
2answers
36 views

Can C4.5 handle continuous attributes?

I'm trying to play with the breast cancer data available through UCI: https://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/wdbc.data When trying to classify the data ...
1
vote
1answer
34 views

10-fold cross-validation (high variation)

I am using 10-fold validation method to validate my model. I am using CART model and my sample size $\approx$ 50. Features $\approx$ 9. The 10-fold validated accuracy (averages) is about 76%. However, ...
1
vote
0answers
58 views

time varying continuous covariate with one single binary outcome

Despite having only a single binary outcome for each ID, there are multiple correlated measurements for the same test for each ID at different timepoints. The individual ID´s are obviously ...
0
votes
0answers
8 views

Tree classifier or nested model?

I am new to statistics and was wondering what the right kind of model to use for the following scenario. I have two sets of continuous observations A and B for 50 samples. These 50 samples are ...
0
votes
1answer
23 views

longitudinal dataset with multiple binary dependent variables

I have a dataset where a test (a continuous variable) is administered every 3 months for 2 years for most of the participants (approx 25% have one or two missing scores). There are further ...
1
vote
1answer
80 views

How to evaluate the goodness of fit for survial functions

I am a newcomer to survival analysis, although I have some knowledge in classification and regression. For regression, we have MSE and R square statistics. But how we can say that survival model A ...
0
votes
0answers
17 views

Fuzzy search in the database

Did anyone have experience with fuzzy search in database? Let's consider some type of Akinator. At each moment the idea is to choose the question which would split all possible (remaining) final ...
0
votes
2answers
34 views

Analyses of mixed variables

I'm relatively new to statistics, and am currently working some data collected as a part of an interview survey. I have a response variable in ordinal form, which mostly looks into people's ...
2
votes
0answers
16 views

Continuous variable evaluation in decision trees

I was going through the C4.5 and ID3 algorithms used to construct a decision tree. Was wondering if there is an efficient way to compute information gain from a continuous variable (during the step ...
1
vote
0answers
5 views

Dependent predictors with converse effects on the target

I am trying to create a predictive model for marketing in the natural gas field. The model is supposed to guess how probable it is to make a contract in that particular building given many internal ...
0
votes
0answers
12 views

Using all the data for a decision tree prediction

When doing regression analysis with training and test sets, I would fit the model to the training data. When happy with the model fit I'd then run it against the test/holdout set to get a true ...
0
votes
0answers
15 views

What statistical test(s) should I run to analyse a directed-tree like data?

Suppose I have data of a kind of binary tree format. There are three levels in the three, thus four different leaf nodes. I would like to find the optimal path from the root node to one of the leaf ...
0
votes
0answers
26 views

VC-Dimension of n-node binary decision tree in N-dimension feature space

Given input feature space $\mathcal{X} =\{0, 1\}^N$ and output label space $\mathcal{Y}=\{0,1\}$ , prove that the VC-dimension of a binary decision tree with $n$ nodes is in $O(n\text{log}N)$. I've ...
2
votes
0answers
61 views

How to split a decision tree when information gains of all attributes are zero?

The textbook tells us that we should choose an attribute with the maximum information gain to split a decision tree. My question is what if all information gains are zero? Should we stop splitting or ...
0
votes
0answers
30 views

Query on Quants

I am working on decision trees for the first time at job. I have done lot of research on CHAID and CART algorithms but find different answers to a very simple question given below : What kind of ...
2
votes
0answers
18 views

mob model tree algorithm

I am trying to figure out the inner workings of the mob function in the party package. I can't figure out how the splitting variable is selected when it is a categorical variable. In the publications ...
0
votes
0answers
7 views

number of nodes in an unpruned decision tree

What is the number of nodes in an unpruned decision tree that is trained using n samples and that grows until there is only one sample in each leaf? I would like to know if there is a formula to ...
0
votes
0answers
39 views

Sample Weights for classification using Gradient-Boosted trees?

How can "weights" be given to different samples according to their relative importance while using Gradient boosted decision trees for classification? How does the ...
4
votes
1answer
87 views

Are Auto-Associative Regression Trees Distinct from Auto-Regressive Trees?

After some reading in the field I was confused as to whether these two models are distinct or really the same. I'm just looking for a simple yes/no with a brief explanation. Note that ...
0
votes
1answer
39 views

rpart complexity parameter confusion

I'm a little bit confused on the calculation for CP in the summary of an rpart object. Take this example ...
0
votes
0answers
49 views

How are CP (Cost Complexity) values calculated in RPART (or decision trees in general)

From what I understand, the cp argument to the rpart function helps pre-prune the tree in the same way as the minsplit or minbucket arguments. What I don't ...
0
votes
0answers
29 views

feature selection for longitudinal data

I have a longitudinal data which looks like this. Number of time points are different for each ID. Y is the binary response variable (take values 0 & 1) and X1-X20 are either continuous or ...
1
vote
1answer
30 views

Obtaining easy-to-interpret decision tree

I am trying to create a decision-tree out of a number of attributes, where there are only two final classes and the classes are highly unbalanced (Class 1: 95.5%; Class 2: 4.5%). The idea is to ...
1
vote
1answer
49 views

Post hoc selection of important features in random forest?

I want to guarantee a parsimonious random forest (few features used). What are methods to do this? It was suggested to me to get the feature importance after the model was created, and then create a ...
0
votes
0answers
22 views

Considering non-i.i.d. covariates in random forests

Random forests are theoretically funded on the assumption that the data are i.i.d. realizations from a multivariate random vector $(X_1, \ldots, X_p, Y)$. Does it make sense to use random forests (for ...
1
vote
0answers
33 views

categorical feature ranking

I would like to rank categorical features by the order or importance in a classification/regression setting. Input There are two features, which are survey questions: "how is your mood?": four ...
0
votes
0answers
15 views

Minimum population for decision trees and association rules

Hi I'm quite new to this and I'm playing around with R and Microsoft's SSAS. Does anyone have a rule of thumb how big a data set has to be for association rules and decision trees to be statistically ...
0
votes
1answer
31 views

A question on no. of training examples and decision trees

I have a set of around 200,000 training instances. Each training instance consists of an attribute called $duration$, a discrete integer type and a time series of floating-point values, in form of a ...
0
votes
0answers
23 views

On population variable importance

Consider we run a random forest on $n$ independent realizations of a random vector $(X_1,X_2,X_3,Y)$ assuming $Y$ is a numerical response variable. Let $f$ be the best theoretical classifier defined ...
0
votes
0answers
16 views

How does RPART pick a splitter when there are at least 2 splits having maximal information gain?

I'm not sure the best way to explain this, so let me give an example that motivates my question. I have tried reading the RPART manual, documentation, and its code, but I have not been able to resolve ...
0
votes
0answers
6 views

Design a feature with time and presence information

Context: I am working on a decision tree classifier, trying to classify businesses as to whether they are likely to have an event occur (default) in the next 90 days. One input I get is whether, and ...
0
votes
0answers
26 views

Evaluating mean-squared error

Hello I am running a Regression Tree experiment. I am new to Regression Trees, and I am using Mean Squared error to test my tree. I am confused because I am getting a large Mean Squared Error but I ...
2
votes
1answer
108 views

Why should all Cross-Validation results be higher than the result on the test dataset?

Sorry, I'm not an expert and my question could be fundamentally wrong. I've read this interesting question because I also was wondering whether to train the model again after cross-validation. Now, ...
1
vote
1answer
39 views

Is there such thing as correlation trees? Clustering rows of X based on correlation between A and B

I have been searching for several days for a method that fits this description, though cannot find one. I'm pretty sure it must exist. The problem (short version): I'd like to run something like a ...
2
votes
2answers
32 views

Decision Trees on training data

Wouldn't any decision tree trained on a training data set have no errors in classification? In other words, wouldn't every data point be classified correctly in the training data set? How would this ...
0
votes
0answers
27 views

Misclassification Rate

The misclassfication rate in decision trees is defined as $$1-\max_{j} p_{j}$$ Suppose we want to classify people into republicans and democrats. In the training data, $\hat{p}_1 = 0.7$ (probability ...
1
vote
0answers
8 views

Comparing and contrasting classification trees over the years

At work I have decided to use a classification tree to do some analysis on the people at the bottom decile for hourly pay. Mainly for inferential reasons. My datasets are broken up in years, I have ...
1
vote
0answers
35 views

In CHAID, shouldn't we merge categories when p<alpha rather than p>alpha?

In CHAID, the categories are merged when P>alpha in the first step. BUT Since CHAID uses Chi-square statisitic, if p-value < alpha, we reject the null ( Independence), hence, meaning the ...
1
vote
1answer
24 views

What methods can be used to transform data?

I am solving a binary classification problem with 4 predictor variables. The variables didn't seem to be linearly separable. I have used Neural Networks and Kernel SVM which work and give desired ...
0
votes
0answers
28 views

Down-sampling with building models (specifically random forests)

I was wondering if anyone had ever used down-sampling to build random forests with data that has unbalanced classes. Basically down-sampling samples (with replacement) x*min from the population where ...
0
votes
0answers
27 views

building a decision tree after association rules

I built association rules using R Arules package. Then I filtered those rules and kept rules that have a specific variable on RHS. That variable is my Y variable. In entire dataset, my variable occurs ...
0
votes
0answers
38 views

Improving sentence segmentation in NLTK

I have been looking into problem of sentence segmentation lately. I have been referring to NLTK's book for this purpose. I followed their procedure to segment sentences presented here: ...
3
votes
0answers
84 views

Why will a random forest not outperform a regression tree?

I have a training dataset with a binary response variable, 6 independent variables, and 21,000 observations. I've fit both an ordinary regression tree and a random forest (mtry = 2, ntree = 2000) and ...
0
votes
0answers
72 views

Decision tree in R

I am new to machine learning in R. This is my data set. ...