'Classification And Regression Trees'. CART is a popular data mining technique.
0
votes
0answers
11 views
checking bi-variate relations in predictive model
Friends,
I am using decision tree and logistic regression for prediction purposes (my dependent variable is a binary variable). I am just wondering whether I need to check chi-square (for categorical ...
0
votes
0answers
18 views
Question about classification problem (in particular curse of dimensionality) [closed]
**A scientist must analyze a large microarray, in which a set of
unit must be classified into two classes on the basis of a very high number of genes (Variables). Because the membership of each unit ...
0
votes
0answers
21 views
how to train model with features have low variance in train set
Assume that I trained a nonlinear model , one feature of the training data has very low variance, because of this, the same feature of the test could be quite different, at least in scale, from the ...
1
vote
1answer
48 views
Can a Decision Tree handle a column which is an array or strings?
I have this dataset where one of the columns (features) is an array of delays codes. Sometimes the array has got 1 single code and sometimes up to 5 codes. The codes can appear just once in the array ...
0
votes
0answers
15 views
How to improve XGBoost (or tree ensembles) results for low values observations in a regression problem?
I built a regression model to predict prices from a set of attributes. I chose XGBoost approach to train. After training I plotted the distribution of the observed values against the distribution of ...
0
votes
0answers
6 views
Decision tree probability score in MATLAB
How does 'predict' function in MATLAB compute probability scores for predictions for binary classification trees?
4
votes
1answer
52 views
Can I combine many gradient boosting trees using bagging technique
Based on Gradient Boosting Tree vs Random Forest . GBDT and RF using different strategy to tackle bias and variance.
My question is that can I resample dataset (with replacement) to train multiple ...
0
votes
0answers
20 views
Data standardization or normalization in GBDT [duplicate]
Is it necessary to do data normalization(standardization) before using gbdt?what effect does it have if I don't do that proprecessing?
0
votes
1answer
37 views
CART on Timeseries Forecasting
I've read in a few articles where it was talked about using CART for timeseries forecasting and anomaly detection. However, I would want to remove the Seasonal and Trend noise in my temporal data. I'...
0
votes
0answers
9 views
Bayesian Additive Regression Trees - model assumptions?
BART builds on regression and classification tree models, and you can use it for continuous and binary outcomes (=probit). See Chipman 2010 for details.
With normal regression methods there are a ...
2
votes
0answers
18 views
Calculating Probability for Decision Tree Model
I came across calculation of probability for a decision tree model - which I do not understand.
As I plan to do CEA of some health interventions I would not like to mess it up.
The used method (...
1
vote
0answers
15 views
Adjusted R-squared for tree-based models
How can I use an evaluation metric like adjusted R-squared to evaluate tree-based models? It's not clear to me, since adjusted R-squared accounts for the number of predictors included in a given model,...
0
votes
1answer
23 views
Ctree - concerning the splitting criteria
I have a technical question concerning the choice of the splitting criteria for the recursive partitioning.
Having selected the most significant variable, I would like to know why the optimal ...
-3
votes
0answers
25 views
In a decision tree, what do 'samples' mean?
On the decision tree diagram below, what do the 'samples' mean and how do they relate to the 'value' line that follows?
0
votes
0answers
28 views
Gradient Boosted Regression - decide number of trees?
By adding arbitrarily many trees, seems like the $R^2$ value can be as close to 1.0 as we want.
This doesn't seem correct. How do we determine the optimal number of trees? Should I use a form of ...
0
votes
0answers
10 views
C&RT Rule Set missing rules?
I configured a C&RT 2-class classifier model from a dataset and then saved the results to a table that I exported to a server running SAS. I used "Generate->Rule Set" on the C&RT model to see ...
1
vote
1answer
41 views
Regression Trees: how to split if node has 2 samples
Sorry, but this is not a general question, so i am going to be as specific as i can. I have searched a lot, however i cant consider the following case in regression trees:
The following picture shows ...
0
votes
0answers
40 views
How to make 65 clusters ? Is k-mean good algorithm to do this?
I am trying to segment customers based on demographic, behavioral, lifestyle etc into 60-65 segments inline with Claritas Prizm segments Link1 Link2
I have 1 million records and 264 variables. ...
0
votes
1answer
11 views
Account for small but significant categories in model
I want to model participation to a campaign.
I have ~200 variables for ~100k observations.
Many variables are categorical and I often found high participation rates in smaller categories, for ...
0
votes
0answers
23 views
What happens when the feature importance plot is dominated by only one feature?
I got a feature importance plot from my gbm model, where one of the feature shows a very high value of feature importance as compared to the other variables. Will that be affecting my predictions in a ...
0
votes
0answers
36 views
Options for Plotting distance Matrix
(Apologies if my terminology is a bit "off", I'm diving into R for a project on late Medieval Devotional Calendars without much background knowledge.)
I'm trying to generate a plot, essentially a ...
0
votes
1answer
19 views
How to extract the split points of mob() [closed]
In rpart I can simply extract the split points of the tree using
...
1
vote
1answer
26 views
Does decision tree need to use the same feature to split in the same layer?
I know in decision tree, we select features which maximize information gain (IG) to split data. My question is that, does such selections need to be the same in the same layer?
Suppose data has ...
0
votes
0answers
15 views
Estimation parameters and global optimum
I have used some approaches to model my data. One of these methods is the sequential estimation approach of the tree-kind models. I see that the method estimated the model parameters very, very well. ...
0
votes
0answers
16 views
Spliting data set that didn't come from same source
I have a data set consisting of features extracted from audio files of individuals with parkinson's disease.
In this set, the training set features were extracted approximately 40 individuals, each ...
1
vote
0answers
16 views
Which would be a good learning set for a Decision Tree of rugby players?
The problem
Given a set of some Instagram users, use a Decision Tree Classifier to understand if either they're rugby players or not.
My approach
I have some perplexities in the learning set. So ...
0
votes
0answers
58 views
What are the final predictions in tree based models?
I always thought that there are only two ways the predictions from the final leaf are extracted in a tree based model:
For regression problems take the average of the continuous variable to be ...
3
votes
1answer
80 views
What is Oblivious Decision Tree and Why?
I read the catboost paper and they mentioned the trees are oblivious decision tree. What is the definition of oblivious decision tree? I found two possible candidates but not sure if they are the ...
0
votes
0answers
12 views
Decision Tree Regression: Target in [0, 1]
I'm trying to use Random Forest for regression, with targets between 0 and 1. Does the target range not matter for decision trees, since the average target at the leaves would always be in [0,1]?
0
votes
0answers
62 views
How to retrieve an underlying mathematical model from the decision tree or random forest? [duplicate]
I wonder if it is possible to retrieve something similar to regression equation from the trained decision tree or random forest for the regression problem?
In particular, I want to know how much each ...
1
vote
0answers
98 views
Understanding regularization in xgboost
A general loss function is:
\begin{split}\text{obj} = \sum_{i=1}^n l(y_i, \hat{y}_i^{(t)}) + \sum_{i=1}^t\Omega(f_i) \\
\end{split}
which is prediction cost + regularization cost
A decision tree is ...
0
votes
0answers
21 views
Regression tree with cross-entropy loss
I need to predict the categorical distribution with the decision tree.
Formally there is a map $ f: X \rightarrow \{(p_1, ..., p_k): p_i \geq 0, \, \text{and} \sum_i p_i = 1\}$ defined on the given ...
0
votes
0answers
24 views
In which cases can SVM totally fail but Decision trees/Multilayer Perceptrons succeed in classification?
I have been working with a binary classification problem recently. The dataset contained 20 features.
I used SVM with different parameter settings and got 50% accuracy (i.e. all predictions are ...
0
votes
0answers
29 views
Boosting - learners with weight?
I'm trying to understand the Boosting method and I'm very confused by what I see on the internet as there are different explanations and I'm not sure some of them are true.
Here's what I've ...
0
votes
1answer
30 views
Writing the equation of boosted regression trees model [closed]
This is the formula for fitting a boosted regression trees from gbm package:
...
0
votes
0answers
14 views
Rule based ISLE Ensemble Generation
I come through a algorithm ISLE Ensemble Generation in machine learning. The following is the steps given in Elements of Statistical Learning:
But I am unable to apprehend it and implement it in ...
1
vote
1answer
53 views
Using trees for conditioned linear regression
I want to explore using tree methods to condition a linear regression. Let's say the baseline regression is of the form $y = \beta x + \epsilon$.
In addition, I have conditioning data, a vector $c \...
3
votes
1answer
113 views
Entropy Impurity, Gini Impurity, Information gain - differences?
I'm trying to understand the theory behind decision trees (CART) and especially the scikit-learn implementation.
CART constructs binary trees using the feature and threshold that
yield the ...
0
votes
0answers
13 views
Random Forest and Decision Trees as Variable Selection methods [duplicate]
I'm doing some variable selection task on a dataset with 130 columns(both categorical and numerical) and 15,000 rows.
I read that random forest and decision tree can be applied for variable ...
1
vote
0answers
12 views
Classification and regression trees: what is a sensible value for the minimum number of observations in terminal nodes
Here I'll borrow some parameter names from the R rpart package for illustration.
The most important hyperparameter for pruning a CART tree is the cost-complexity ...
0
votes
2answers
44 views
classification tree with rpart()
I'm using the rpart() to build a classification tree using R. I have no experience in this topic... Anyway, I started with the full model, and then I used the varImp() from "caret" to drop some ...
1
vote
0answers
18 views
Use GBM to find the optimum configuration of variables given certain conditions
I have a data set containing historical data of a SAG Ball Mill There are two types of variables: Mineral Characteristics, and Mill Paremeters. Mineral Characteristics are given and can not be ...
0
votes
0answers
7 views
Can regression trees or any other model be used to provide equation of separating hyperplane just like SVMs?
I am working on image segmentation. Suppose I have various spheres derived from the maximal ball algorithm. I want to generate a hyperplane that fits in the best way. Which methods other than SVM ...
1
vote
1answer
57 views
Decision tree vs linear SVM
Is it correct to say that, a difference between decision trees and linear SVMs is that the hyperplanes used by the decision trees are perpendicular to axis?
1
vote
1answer
30 views
CART Decision Tree Learning when some variables are ranges of continuous values
Could anyone kindly give some practical advice on how to deal with predictors of a range of values?
For instance, I want to predict $Y$ based on features $X_{i}, \ i=1,2,...,N$. Some $X_{i}$s are of ...
1
vote
0answers
38 views
CART selection and *deselection* classification tree
I came across a line in Peterson, et al. (2016) that says:
The specific settings applied in the rpart procedure ensured that only the largest subgroup would be ...
0
votes
0answers
36 views
Can't Validate Model Result
This is my first question and I am still new to data science and statistics, so please forgive me if this is a dumb or ill-posed question. I have two ensemble tree classifier models built on the same ...
1
vote
0answers
37 views
Performance of Conditional Inference regression Trees updating the influence function at each node
My goal is to compare the performance of $2$ models of trees using the Conditional Inference tree framework described in (ctree: Conditional Inference Trees), I am following the Partykit 2018.
...
12
votes
1answer
903 views
Depth of a decision tree
Since the decision tree algorithm split on an attribute at every step, the maximum depth of a decision tree is equal to the number of attributes of the data.
Is this correct?
1
vote
0answers
16 views
CRAN Tree Package [closed]
I am using the tree package in R.
What does (prune object)$k=-Inf mean? I mean what is the difference between k=0 and k=-Inf (k is the complexity parameter).
...
