Tagged Questions

1
vote
0answers
50 views

Deciding attributes for decision trees

I'm a complete beginner when it comes to R and decision trees, but I was asked to take a look at this to see if this was a viable solution for my data. So please excuse me if I say completely wrong. ...
0
votes
0answers
30 views

Is there a theoretical basis for the shrinkage used in Boosted Regression Trees?

In Gradient Boosted Regression Trees, a shrinkage $\nu$ is often applied as: $$ f_t(x) \leftarrow f_{t-1}(x) + \nu h(x)$$ where $h$ is the regression tree learned by fitting the tree to the gradient. ...
1
vote
1answer
104 views

Building the dataset for Random Forest training procedure

I should use the bagging (bootstrap aggregating) technique in order to train a random forest classifier. I read here the description of this learning technique, but I have not figured out how I ...
1
vote
0answers
61 views

Determining optimal height for regression tree

I have a data set of approximately 400,000 records (for those of you who know, the data set is the one provided by yahoo for their yahoo learning to rank challenge). From this data set I learn a ...
2
votes
1answer
201 views

What is “feature space”?

What is the definition of "feature space"? For example, When reading about SVMs, I read about "mapping to feature space". When reading about CART, I read about "partitioning to feature space". I ...
4
votes
0answers
312 views

Would a Random Forest with multiple outputs be possible/practical?

Random Forests (RFs) is a competitive data modeling/mining method. An RF model has one output -- the output/prediction variable. The naive approach to modeling multiple outputs with RFs would be to ...
4
votes
1answer
78 views

Is there a method for constructing decision trees that takes account of structured/hierarchical/multilevel predictors?

Is there a method for constructing decision trees that takes account of structured/hierarchical/multilevel predictors, that would allow me to impose domain knowledge or constraints on interactions for ...
3
votes
1answer
122 views

Should I use decision trees to predict user preferences?

I'm designing a web service that will predict and recommend new items a user might like based on their expressed preferences on previous items (simple thumbs up/down interface). I was told to look ...
5
votes
4answers
469 views

Benchmark dataset for decision tree algorithm

I'm implementing a decision tree algorithm, and I'd like to get a feel for how it performs relative to other implementations. Can anyone recommend popular datasets for training and testing decision ...
1
vote
0answers
48 views

Relationship between the characteristics of training data set and built decision tree

I have a training data set for a binary classification problem. There exist two possible scenarios, one is that all of the training data set are labeled as positive; another one is that the training ...
1
vote
0answers
201 views

Obtaining resampling based estimates of prediction error in boosted regression tree model

I try to use the gbm.fit() function for a boosted regression tree model implemented in the R package gbm. To investigate e.g., the bootstrapped prediction error and ...
4
votes
2answers
117 views

Incorporating seasonality into CART models

The problem I am trying to solve it predicting sales for an item for the next $n$ weeks. Obviously, seasonality is a major factor for such predictions. If we use a time series based model, then we ...
4
votes
3answers
610 views

Are decision trees almost always binary trees?

Nearly every decision tree example I've come across happens to be a binary tree. Is this pretty much universal? Do most of the standard algorithms (C4.5, CART, etc.) only support binary trees? From ...
22
votes
1answer
3k views

Conditional inference trees vs traditional decision trees

Can anyone explain the primary differences between conditional inference trees (ctree from party package in R) compared to the ...
4
votes
1answer
239 views

Do infrequent examples screw up classifiers? If so, when is it okay to remove the infrequent examples from the data?

It's hard to think of a more eloquent way of phrasing this question - I'm basically wondering if a classifier trained on data where examples of some of the classes are infrequent/rare would be a bad ...
6
votes
2answers
348 views

Are there any libraries available for CART-like methods using sparse predictors & responses?

I'm working with some large data sets using the gbm package in R. Both my predictor matrix and my response vector are pretty sparse (i.e. most entries are zero). I was hoping to build decision trees ...
3
votes
1answer
1k views

What is the difference between empirical variance and variance?

As far as I know variance is calculated as $$\text{variance} = \frac{(x-\text{mean})^2}{n}$$ while $$\text{Empirical Variance} = \frac{(x-\text{mean})^2}{n(n-1)} $$ Is it correct? Or is there ...
13
votes
5answers
972 views

Alternatives to classification trees, with better predictive (e.g: CV) performance?

I am looking for an alternative to Classification Trees which might yield better predictive power. The data I am dealing with has factors for both the explanatory and the explained variables. I ...
8
votes
4answers
1k views

What is the weak side of decision trees?

Decision trees seems to be a very understandable machine learning method. Once created it can be easily inspected by a human which is a great advantage in some applications. What are the practical ...
9
votes
2answers
1k views

How does random forest generate the random forest

I am not an expert of random forest but I clearly understand that the key issue with random forest is the (random) tree generation. Can you explain me how the trees are generated? (i.e. What is the ...