Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

'Classification And Regression Trees'. CART is a popular data mining technique.

1
vote
2answers
53 views

multicollinearity resulting in high variance

Section 8.7.1 of Elements of Statistical Learning talks about high variance in a classification tree due to high correlation between features. What is the intuition behind this? I would think that ...
0
votes
0answers
16 views

How can I run a decision tree algorithm with a specific hierarchy of variables and with many missing values?

I asked students in learning groups what their biggest learning problem was "today" for each learner. The biggest problem could either be "motivational" (=motivation problem) or cognitive (="knowledge ...
0
votes
1answer
31 views

Classification Tree or Regression Tree?

I have time series data: students that learned in groups for minimum 3 times and maximum 10 times and for each learning group session had to state if they faced a motivational OR a cognitive problem, ...
1
vote
1answer
55 views

optimal decision tree np-hard

Reading Elements of Statistical Learning and it says that decision trees are often constructed using greedy algorithms because it is computationally infeasible to create an optimal decision tree. ...
0
votes
0answers
9 views

Regression vs. Decision trees

it is not clear to me when it is appropriate to use a decision tree approach vs. a conventional regression model. Let's take the example of Titanic survivors as shown here https://en.wikipedia.org/...
0
votes
1answer
45 views

Decision tree without the “tree”

I would like to construct something like a decision tree. However, instead of using "recursive partitioning" to build a tree, I would like to find an optimal set of "global" splits. For example, in a ...
0
votes
1answer
18 views

Merge one label with one information for classification problem or multi-label classification

I want to build a model to support decision making in order to propose or not loan insurance to clients. Because sometimes clients asking loan and loan insurance have less chance to have their loan ...
2
votes
1answer
17 views

what if a decision tree does not result in leaves with one class each?

A decision tree can result in leaf nodes that have samples from multiple classes. Is the algorithm at that point to simply vote on the class?
0
votes
0answers
9 views

Is pseudo-r2 enough to validate a CART?

I run some Classification and Regression Trees (CARTs) and computed the pseudo-$R^2$ from McFadden. Is that enough to validate the trees or do I need some other test to be sure there is no overfitting?...
0
votes
0answers
13 views

Difference between Random Forest and Random Subspaces/Patches

When fitting a Random Forest model, a subset of the features is randomly considered at the splitting of each node. E.g., if $p$ is the number of features, then at each node in each tree, $\sqrt{p}$ ...
0
votes
0answers
18 views

Which algorithms for mixed type datasets (binary classification)?

I am new to machine learning and I am trying to implement a model for a binary classification problem (output class 0 or class 1), and wondering which algorithms I should consider, since my dataset is ...
1
vote
0answers
20 views

What is the relation between minimum instances per node and max depth?

In bagging and boosting models like random forest and xgboost we have hyper-parameters like minimum instances per node and max depth. If max depth is high the minimum instances per node will be less ...
3
votes
1answer
44 views

Which model for feature importances?

When wanting to find which features are the most important in a dataset, most people use a linear model - in most cases an L1 regularized one (i.e. Lasso). However, tree based algorithms have their ...
2
votes
0answers
22 views

True or False? Not testing data is needed for CART if there is not future prediction

I am analysing a data set with a relatively a small sample size. For the nature of my data, it is considered already quite large, but for its statistical power it is just in the lower limit. I sampled ...
0
votes
0answers
8 views

Small sample size for partitioning and evaluating a CART

I have a data set of 130 samples. If I partition the data 70/30 % to run a conditional tree, CART, and then evaluate it, the results are different than if I run the 100% of my data in the CART. When I ...
0
votes
0answers
21 views

Decision Tree with unbalanced dataset in SAS

I have a dataset with a binary target variable. This variable is highly imbalanced i.e. the # of True case is ~1% and # of False cases is ~99% The other limitation I have is that I can only use ...
1
vote
0answers
31 views

Dealing with unbalanced data in an explanatory model with decision tree

I am doing binary classification with decision tree, and it aims to find out what features matter the most with the data we have, so I need interpretability more than predictability. It is like ...
1
vote
0answers
26 views

How to use random forest for regression after it is trained

I don't understand how to work with a random forest regressor after it is trained. I read and coded some tutorials about regression with random forests in Python with scikit but I don't understand how ...
0
votes
0answers
10 views

Scale response variable y in random forest or gradient boosted trees for regression == scale prediction?

Suppose we are fitting a random forest or gradient boosted tree model for regression on y. We first fit the model. Later on, we realized we need to fit y at another different scale, for example, a <...
0
votes
0answers
23 views

How to decide which dataset will be difficult for the decision tree algorithm to learn?

we have three datasets MONK-1, MONK-2 and MONK-3 over a six-attribute discrete domain. The attributes a1,a2,a3,a4,a5,a6 may take the following values: a1 ∈{1,2,3} a2 ∈{1,2,3} a3 ∈{1,2} a4 ∈{1,2,3} a5 ...
1
vote
2answers
38 views

How NULLs in numerical variables are treated in tree-based models?

I understand that in tree-based models (CART, Gradient boosted trees, etc.), NULLs (i.e., NaN) in categorical variables can be treated as a separated category, while making node splits. However, how ...
0
votes
0answers
8 views

What a convex Precision-Recall curve means for training dataset?

Situation I have trained a GBDT model(gradient-boosted decision tree, a tree ensemble model) with a training dataset, and when I calculate PR curve on the same training set, it looks convex: For ...
0
votes
1answer
54 views

Is there any structure which combines decision tree and continuous variables?

The starting point is intuitive I guess: I want to do a decision tree type of algorithm, but some of the variables are continuous, e.g. numbers instead of quantities like ages. I cannot afford the ...
0
votes
0answers
17 views

Prediction with categorical and continuous Variables

I want to predict the result of a match in a video game (win or loose). It's 5 players against 5 players game, who each plays a specific character. I have : the ID of each character (there are 150 ...
0
votes
0answers
23 views
1
vote
0answers
18 views

Boosted regression trees update of residuals

In my book on statistical learning, an algorithm for boosting for regression trees is described. They have the main step of the algorithm as: $\hat{f}(x) \leftarrow f(x) + \lambda \hat{f}^b(x)$, ...
0
votes
0answers
8 views

Number of datas regarding the number of features

I'm trying to make predictions on a video game. The game is a 5v5 battle with champions. There are about 200 different champions. My input is a vector of 10 champion IDs (5 IDs for each team) and my ...
4
votes
1answer
101 views

On what types of datasets do tree-based models not do well?

Are there examples where splitting on the best feature/threshold combination is not actually the best way to split the tree, and that better results could be got by choosing a different feature but ...
2
votes
2answers
117 views

Random Forests and Information gain

Suppose you are building random forest model, which split a node on the attribute, that has highest information gain. In the below image, select the attribute which has the highest information gain? ...
0
votes
0answers
13 views

checking bi-variate relations in predictive model

Friends, I am using decision tree and logistic regression for prediction purposes (my dependent variable is a binary variable). I am just wondering whether I need to check chi-square (for categorical ...
0
votes
0answers
30 views

how to train model with features have low variance in train set

Assume that I trained a nonlinear model , one feature of the training data has very low variance, because of this, the same feature of the test could be quite different, at least in scale, from the ...
1
vote
1answer
88 views

Can a Decision Tree handle a column which is an array or strings?

I have this dataset where one of the columns (features) is an array of delays codes. Sometimes the array has got 1 single code and sometimes up to 5 codes. The codes can appear just once in the array ...
0
votes
0answers
25 views

How to improve XGBoost (or tree ensembles) results for low values observations in a regression problem?

I built a regression model to predict prices from a set of attributes. I chose XGBoost approach to train. After training I plotted the distribution of the observed values against the distribution of ...
0
votes
0answers
10 views

Decision tree probability score in MATLAB

How does 'predict' function in MATLAB compute probability scores for predictions for binary classification trees?
4
votes
1answer
88 views

Can I combine many gradient boosting trees using bagging technique

Based on Gradient Boosting Tree vs Random Forest . GBDT and RF using different strategy to tackle bias and variance. My question is that can I resample dataset (with replacement) to train multiple ...
0
votes
0answers
24 views

Data standardization or normalization in GBDT [duplicate]

Is it necessary to do data normalization(standardization) before using gbdt?what effect does it have if I don't do that proprecessing?
0
votes
1answer
47 views

CART on Timeseries Forecasting

I've read in a few articles where it was talked about using CART for timeseries forecasting and anomaly detection. However, I would want to remove the Seasonal and Trend noise in my temporal data. I'...
0
votes
0answers
10 views

Bayesian Additive Regression Trees - model assumptions?

BART builds on regression and classification tree models, and you can use it for continuous and binary outcomes (=probit). See Chipman 2010 for details. With normal regression methods there are a ...
2
votes
0answers
26 views

Calculating Probability for Decision Tree Model

I came across calculation of probability for a decision tree model - which I do not understand. As I plan to do CEA of some health interventions I would not like to mess it up. The used method (...
1
vote
0answers
20 views

Adjusted R-squared for tree-based models

How can I use an evaluation metric like adjusted R-squared to evaluate tree-based models? It's not clear to me, since adjusted R-squared accounts for the number of predictors included in a given model,...
0
votes
1answer
28 views

Ctree - concerning the splitting criteria

I have a technical question concerning the choice of the splitting criteria for the recursive partitioning. Having selected the most significant variable, I would like to know why the optimal ...
0
votes
0answers
30 views

Gradient Boosted Regression - decide number of trees?

By adding arbitrarily many trees, seems like the $R^2$ value can be as close to 1.0 as we want. This doesn't seem correct. How do we determine the optimal number of trees? Should I use a form of ...
0
votes
0answers
10 views

C&RT Rule Set missing rules?

I configured a C&RT 2-class classifier model from a dataset and then saved the results to a table that I exported to a server running SAS. I used "Generate->Rule Set" on the C&RT model to see ...
1
vote
1answer
63 views

Regression Trees: how to split if node has 2 samples

Sorry, but this is not a general question, so i am going to be as specific as i can. I have searched a lot, however i cant consider the following case in regression trees: The following picture shows ...
0
votes
0answers
41 views

How to make 65 clusters ? Is k-mean good algorithm to do this?

I am trying to segment customers based on demographic, behavioral, lifestyle etc into 60-65 segments inline with Claritas Prizm segments Link1 Link2 I have 1 million records and 264 variables. ...
0
votes
1answer
11 views

Account for small but significant categories in model

I want to model participation to a campaign. I have ~200 variables for ~100k observations. Many variables are categorical and I often found high participation rates in smaller categories, for ...
0
votes
0answers
44 views

What happens when the feature importance plot is dominated by only one feature?

I got a feature importance plot from my gbm model, where one of the feature shows a very high value of feature importance as compared to the other variables. Will that be affecting my predictions in a ...
0
votes
0answers
149 views

Options for Plotting distance Matrix

(Apologies if my terminology is a bit "off", I'm diving into R for a project on late Medieval Devotional Calendars without much background knowledge.) I'm trying to generate a plot, essentially a ...
0
votes
1answer
32 views

How to extract the split points of mob() [closed]

In rpart I can simply extract the split points of the tree using ...
1
vote
1answer
46 views

Does decision tree need to use the same feature to split in the same layer?

I know in decision tree, we select features which maximize information gain (IG) to split data. My question is that, does such selections need to be the same in the same layer? Suppose data has ...