'Classification And Regression Trees'. CART is a popular data mining technique.
1
vote
2answers
53 views
multicollinearity resulting in high variance
Section 8.7.1 of Elements of Statistical Learning talks about high variance in a classification tree due to high correlation between features. What is the intuition behind this? I would think that ...
0
votes
0answers
16 views
How can I run a decision tree algorithm with a specific hierarchy of variables and with many missing values?
I asked students in learning groups what their biggest learning problem was "today" for each learner. The biggest problem could either be "motivational" (=motivation problem) or cognitive (="knowledge ...
0
votes
1answer
31 views
Classification Tree or Regression Tree?
I have time series data: students that learned in groups for minimum 3 times and maximum 10 times and for each learning group session had to state if they faced a motivational OR a cognitive problem, ...
1
vote
1answer
55 views
optimal decision tree np-hard
Reading Elements of Statistical Learning and it says that decision trees are often constructed using greedy algorithms because it is computationally infeasible to create an optimal decision tree. ...
0
votes
0answers
9 views
Regression vs. Decision trees
it is not clear to me when it is appropriate to use a decision tree approach vs. a conventional regression model. Let's take the example of Titanic survivors as shown here https://en.wikipedia.org/...
0
votes
1answer
45 views
Decision tree without the “tree”
I would like to construct something like a decision tree. However, instead of using "recursive partitioning" to build a tree, I would like to find an optimal set of "global" splits.
For example, in a ...
0
votes
1answer
18 views
Merge one label with one information for classification problem or multi-label classification
I want to build a model to support decision making in order to propose or not loan insurance to clients. Because sometimes clients asking loan and loan insurance have less chance to have their loan ...
2
votes
1answer
17 views
what if a decision tree does not result in leaves with one class each?
A decision tree can result in leaf nodes that have samples from multiple classes. Is the algorithm at that point to simply vote on the class?
0
votes
0answers
9 views
Is pseudo-r2 enough to validate a CART?
I run some Classification and Regression Trees (CARTs) and computed the pseudo-$R^2$ from McFadden. Is that enough to validate the trees or do I need some other test to be sure there is no overfitting?...
0
votes
0answers
13 views
Difference between Random Forest and Random Subspaces/Patches
When fitting a Random Forest model, a subset of the features is randomly considered at the splitting of each node. E.g., if $p$ is the number of features, then at each node in each tree, $\sqrt{p}$ ...
0
votes
0answers
18 views
Which algorithms for mixed type datasets (binary classification)?
I am new to machine learning and I am trying to implement a model for a binary classification problem (output class 0 or class 1), and wondering which algorithms I should consider, since my dataset is ...
1
vote
0answers
20 views
What is the relation between minimum instances per node and max depth?
In bagging and boosting models like random forest and xgboost we have hyper-parameters like minimum instances per node and max depth. If max depth is high the minimum instances per node will be less ...
3
votes
1answer
44 views
Which model for feature importances?
When wanting to find which features are the most important in a dataset, most people use a linear model - in most cases an L1 regularized one (i.e. Lasso).
However, tree based algorithms have their ...
2
votes
0answers
22 views
True or False? Not testing data is needed for CART if there is not future prediction
I am analysing a data set with a relatively a small sample size. For the nature of my data, it is considered already quite large, but for its statistical power it is just in the lower limit. I sampled ...
0
votes
0answers
8 views
Small sample size for partitioning and evaluating a CART
I have a data set of 130 samples. If I partition the data 70/30 % to run a conditional tree, CART, and then evaluate it, the results are different than if I run the 100% of my data in the CART. When I ...
0
votes
0answers
21 views
Decision Tree with unbalanced dataset in SAS
I have a dataset with a binary target variable. This variable is highly imbalanced i.e. the # of True case is ~1% and # of False cases is ~99% The other limitation I have is that I can only use ...
1
vote
0answers
31 views
Dealing with unbalanced data in an explanatory model with decision tree
I am doing binary classification with decision tree, and it aims to find out what features matter the most with the data we have, so I need interpretability more than predictability. It is like ...
1
vote
0answers
26 views
How to use random forest for regression after it is trained
I don't understand how to work with a random forest regressor after it is trained. I read and coded some tutorials about regression with random forests in Python with scikit but I don't understand how ...
0
votes
0answers
10 views
Scale response variable y in random forest or gradient boosted trees for regression == scale prediction?
Suppose we are fitting a random forest or gradient boosted tree model for regression on y. We first fit the model. Later on, we realized we need to fit y at another different scale, for example, a <...
0
votes
0answers
23 views
How to decide which dataset will be difficult for the decision tree algorithm to learn?
we have three datasets MONK-1, MONK-2 and MONK-3 over a six-attribute discrete domain. The attributes a1,a2,a3,a4,a5,a6 may take the following values:
a1 ∈{1,2,3} a2 ∈{1,2,3} a3 ∈{1,2} a4 ∈{1,2,3} a5 ...
1
vote
2answers
38 views
How NULLs in numerical variables are treated in tree-based models?
I understand that in tree-based models (CART, Gradient boosted trees, etc.), NULLs (i.e., NaN) in categorical variables can be treated as a separated category, while making node splits. However, how ...
0
votes
0answers
8 views
What a convex Precision-Recall curve means for training dataset?
Situation
I have trained a GBDT model(gradient-boosted decision tree, a tree ensemble model) with a training dataset, and when I calculate PR curve on the same training set, it looks convex:
For ...
0
votes
1answer
54 views
Is there any structure which combines decision tree and continuous variables?
The starting point is intuitive I guess: I want to do a decision tree type of algorithm, but some of the variables are continuous, e.g. numbers instead of quantities like ages.
I cannot afford the ...
0
votes
0answers
17 views
Prediction with categorical and continuous Variables
I want to predict the result of a match in a video game (win or loose).
It's 5 players against 5 players game, who each plays a specific character.
I have : the ID of each character (there are 150 ...
0
votes
0answers
23 views
1
vote
0answers
18 views
Boosted regression trees update of residuals
In my book on statistical learning, an algorithm for boosting for regression trees is described. They have the main step of the algorithm as:
$\hat{f}(x) \leftarrow f(x) + \lambda \hat{f}^b(x)$,
...
0
votes
0answers
8 views
Number of datas regarding the number of features
I'm trying to make predictions on a video game.
The game is a 5v5 battle with champions. There are about 200 different champions.
My input is a vector of 10 champion IDs (5 IDs for each team) and my ...
4
votes
1answer
101 views
On what types of datasets do tree-based models not do well?
Are there examples where splitting on the best feature/threshold combination is not actually the best way to split the tree, and that better results could be got by choosing a different feature but ...
2
votes
2answers
117 views
Random Forests and Information gain
Suppose you are building random forest model, which split a node on the attribute, that has highest information gain. In the below image, select the attribute which has the highest information gain?
...
0
votes
0answers
13 views
checking bi-variate relations in predictive model
Friends,
I am using decision tree and logistic regression for prediction purposes (my dependent variable is a binary variable). I am just wondering whether I need to check chi-square (for categorical ...
0
votes
0answers
30 views
how to train model with features have low variance in train set
Assume that I trained a nonlinear model , one feature of the training data has very low variance, because of this, the same feature of the test could be quite different, at least in scale, from the ...
1
vote
1answer
88 views
Can a Decision Tree handle a column which is an array or strings?
I have this dataset where one of the columns (features) is an array of delays codes. Sometimes the array has got 1 single code and sometimes up to 5 codes. The codes can appear just once in the array ...
0
votes
0answers
25 views
How to improve XGBoost (or tree ensembles) results for low values observations in a regression problem?
I built a regression model to predict prices from a set of attributes. I chose XGBoost approach to train. After training I plotted the distribution of the observed values against the distribution of ...
0
votes
0answers
10 views
Decision tree probability score in MATLAB
How does 'predict' function in MATLAB compute probability scores for predictions for binary classification trees?
4
votes
1answer
88 views
Can I combine many gradient boosting trees using bagging technique
Based on Gradient Boosting Tree vs Random Forest . GBDT and RF using different strategy to tackle bias and variance.
My question is that can I resample dataset (with replacement) to train multiple ...
0
votes
0answers
24 views
Data standardization or normalization in GBDT [duplicate]
Is it necessary to do data normalization(standardization) before using gbdt?what effect does it have if I don't do that proprecessing?
0
votes
1answer
47 views
CART on Timeseries Forecasting
I've read in a few articles where it was talked about using CART for timeseries forecasting and anomaly detection. However, I would want to remove the Seasonal and Trend noise in my temporal data. I'...
0
votes
0answers
10 views
Bayesian Additive Regression Trees - model assumptions?
BART builds on regression and classification tree models, and you can use it for continuous and binary outcomes (=probit). See Chipman 2010 for details.
With normal regression methods there are a ...
2
votes
0answers
26 views
Calculating Probability for Decision Tree Model
I came across calculation of probability for a decision tree model - which I do not understand.
As I plan to do CEA of some health interventions I would not like to mess it up.
The used method (...
1
vote
0answers
20 views
Adjusted R-squared for tree-based models
How can I use an evaluation metric like adjusted R-squared to evaluate tree-based models? It's not clear to me, since adjusted R-squared accounts for the number of predictors included in a given model,...
0
votes
1answer
28 views
Ctree - concerning the splitting criteria
I have a technical question concerning the choice of the splitting criteria for the recursive partitioning.
Having selected the most significant variable, I would like to know why the optimal ...
0
votes
0answers
30 views
Gradient Boosted Regression - decide number of trees?
By adding arbitrarily many trees, seems like the $R^2$ value can be as close to 1.0 as we want.
This doesn't seem correct. How do we determine the optimal number of trees? Should I use a form of ...
0
votes
0answers
10 views
C&RT Rule Set missing rules?
I configured a C&RT 2-class classifier model from a dataset and then saved the results to a table that I exported to a server running SAS. I used "Generate->Rule Set" on the C&RT model to see ...
1
vote
1answer
63 views
Regression Trees: how to split if node has 2 samples
Sorry, but this is not a general question, so i am going to be as specific as i can. I have searched a lot, however i cant consider the following case in regression trees:
The following picture shows ...
0
votes
0answers
41 views
How to make 65 clusters ? Is k-mean good algorithm to do this?
I am trying to segment customers based on demographic, behavioral, lifestyle etc into 60-65 segments inline with Claritas Prizm segments Link1 Link2
I have 1 million records and 264 variables. ...
0
votes
1answer
11 views
Account for small but significant categories in model
I want to model participation to a campaign.
I have ~200 variables for ~100k observations.
Many variables are categorical and I often found high participation rates in smaller categories, for ...
0
votes
0answers
44 views
What happens when the feature importance plot is dominated by only one feature?
I got a feature importance plot from my gbm model, where one of the feature shows a very high value of feature importance as compared to the other variables. Will that be affecting my predictions in a ...
0
votes
0answers
149 views
Options for Plotting distance Matrix
(Apologies if my terminology is a bit "off", I'm diving into R for a project on late Medieval Devotional Calendars without much background knowledge.)
I'm trying to generate a plot, essentially a ...
0
votes
1answer
32 views
How to extract the split points of mob() [closed]
In rpart I can simply extract the split points of the tree using
...
1
vote
1answer
46 views
Does decision tree need to use the same feature to split in the same layer?
I know in decision tree, we select features which maximize information gain (IG) to split data. My question is that, does such selections need to be the same in the same layer?
Suppose data has ...
