Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Join us in building a kind, collaborative learning community via our updated Code of Conduct.

'Classification And Regression Trees'. CART is a popular data mining technique.

0
votes
0answers
11 views

checking bi-variate relations in predictive model

Friends, I am using decision tree and logistic regression for prediction purposes (my dependent variable is a binary variable). I am just wondering whether I need to check chi-square (for categorical ...
0
votes
0answers
18 views

Question about classification problem (in particular curse of dimensionality) [closed]

**A scientist must analyze a large microarray, in which a set of unit must be classified into two classes on the basis of a very high number of genes (Variables). Because the membership of each unit ...
0
votes
0answers
21 views

how to train model with features have low variance in train set

Assume that I trained a nonlinear model , one feature of the training data has very low variance, because of this, the same feature of the test could be quite different, at least in scale, from the ...
1
vote
1answer
48 views

Can a Decision Tree handle a column which is an array or strings?

I have this dataset where one of the columns (features) is an array of delays codes. Sometimes the array has got 1 single code and sometimes up to 5 codes. The codes can appear just once in the array ...
0
votes
0answers
15 views

How to improve XGBoost (or tree ensembles) results for low values observations in a regression problem?

I built a regression model to predict prices from a set of attributes. I chose XGBoost approach to train. After training I plotted the distribution of the observed values against the distribution of ...
0
votes
0answers
6 views

Decision tree probability score in MATLAB

How does 'predict' function in MATLAB compute probability scores for predictions for binary classification trees?
4
votes
1answer
52 views

Can I combine many gradient boosting trees using bagging technique

Based on Gradient Boosting Tree vs Random Forest . GBDT and RF using different strategy to tackle bias and variance. My question is that can I resample dataset (with replacement) to train multiple ...
0
votes
0answers
20 views

Data standardization or normalization in GBDT [duplicate]

Is it necessary to do data normalization(standardization) before using gbdt?what effect does it have if I don't do that proprecessing?
0
votes
1answer
37 views

CART on Timeseries Forecasting

I've read in a few articles where it was talked about using CART for timeseries forecasting and anomaly detection. However, I would want to remove the Seasonal and Trend noise in my temporal data. I'...
0
votes
0answers
9 views

Bayesian Additive Regression Trees - model assumptions?

BART builds on regression and classification tree models, and you can use it for continuous and binary outcomes (=probit). See Chipman 2010 for details. With normal regression methods there are a ...
2
votes
0answers
18 views

Calculating Probability for Decision Tree Model

I came across calculation of probability for a decision tree model - which I do not understand. As I plan to do CEA of some health interventions I would not like to mess it up. The used method (...
1
vote
0answers
15 views

Adjusted R-squared for tree-based models

How can I use an evaluation metric like adjusted R-squared to evaluate tree-based models? It's not clear to me, since adjusted R-squared accounts for the number of predictors included in a given model,...
0
votes
1answer
23 views

Ctree - concerning the splitting criteria

I have a technical question concerning the choice of the splitting criteria for the recursive partitioning. Having selected the most significant variable, I would like to know why the optimal ...
-3
votes
0answers
25 views

In a decision tree, what do 'samples' mean?

On the decision tree diagram below, what do the 'samples' mean and how do they relate to the 'value' line that follows?
0
votes
0answers
28 views

Gradient Boosted Regression - decide number of trees?

By adding arbitrarily many trees, seems like the $R^2$ value can be as close to 1.0 as we want. This doesn't seem correct. How do we determine the optimal number of trees? Should I use a form of ...
0
votes
0answers
10 views

C&RT Rule Set missing rules?

I configured a C&RT 2-class classifier model from a dataset and then saved the results to a table that I exported to a server running SAS. I used "Generate->Rule Set" on the C&RT model to see ...
1
vote
1answer
41 views

Regression Trees: how to split if node has 2 samples

Sorry, but this is not a general question, so i am going to be as specific as i can. I have searched a lot, however i cant consider the following case in regression trees: The following picture shows ...
0
votes
0answers
40 views

How to make 65 clusters ? Is k-mean good algorithm to do this?

I am trying to segment customers based on demographic, behavioral, lifestyle etc into 60-65 segments inline with Claritas Prizm segments Link1 Link2 I have 1 million records and 264 variables. ...
0
votes
1answer
11 views

Account for small but significant categories in model

I want to model participation to a campaign. I have ~200 variables for ~100k observations. Many variables are categorical and I often found high participation rates in smaller categories, for ...
0
votes
0answers
23 views

What happens when the feature importance plot is dominated by only one feature?

I got a feature importance plot from my gbm model, where one of the feature shows a very high value of feature importance as compared to the other variables. Will that be affecting my predictions in a ...
0
votes
0answers
36 views

Options for Plotting distance Matrix

(Apologies if my terminology is a bit "off", I'm diving into R for a project on late Medieval Devotional Calendars without much background knowledge.) I'm trying to generate a plot, essentially a ...
0
votes
1answer
19 views

How to extract the split points of mob() [closed]

In rpart I can simply extract the split points of the tree using ...
1
vote
1answer
26 views

Does decision tree need to use the same feature to split in the same layer?

I know in decision tree, we select features which maximize information gain (IG) to split data. My question is that, does such selections need to be the same in the same layer? Suppose data has ...
0
votes
0answers
15 views

Estimation parameters and global optimum

I have used some approaches to model my data. One of these methods is the sequential estimation approach of the tree-kind models. I see that the method estimated the model parameters very, very well. ...
0
votes
0answers
16 views

Spliting data set that didn't come from same source

I have a data set consisting of features extracted from audio files of individuals with parkinson's disease. In this set, the training set features were extracted approximately 40 individuals, each ...
1
vote
0answers
16 views

Which would be a good learning set for a Decision Tree of rugby players?

The problem Given a set of some Instagram users, use a Decision Tree Classifier to understand if either they're rugby players or not. My approach I have some perplexities in the learning set. So ...
0
votes
0answers
58 views

What are the final predictions in tree based models?

I always thought that there are only two ways the predictions from the final leaf are extracted in a tree based model: For regression problems take the average of the continuous variable to be ...
3
votes
1answer
80 views

What is Oblivious Decision Tree and Why?

I read the catboost paper and they mentioned the trees are oblivious decision tree. What is the definition of oblivious decision tree? I found two possible candidates but not sure if they are the ...
0
votes
0answers
12 views

Decision Tree Regression: Target in [0, 1]

I'm trying to use Random Forest for regression, with targets between 0 and 1. Does the target range not matter for decision trees, since the average target at the leaves would always be in [0,1]?
0
votes
0answers
62 views

How to retrieve an underlying mathematical model from the decision tree or random forest? [duplicate]

I wonder if it is possible to retrieve something similar to regression equation from the trained decision tree or random forest for the regression problem? In particular, I want to know how much each ...
1
vote
0answers
98 views

Understanding regularization in xgboost

A general loss function is: \begin{split}\text{obj} = \sum_{i=1}^n l(y_i, \hat{y}_i^{(t)}) + \sum_{i=1}^t\Omega(f_i) \\ \end{split} which is prediction cost + regularization cost A decision tree is ...
0
votes
0answers
21 views

Regression tree with cross-entropy loss

I need to predict the categorical distribution with the decision tree. Formally there is a map $ f: X \rightarrow \{(p_1, ..., p_k): p_i \geq 0, \, \text{and} \sum_i p_i = 1\}$ defined on the given ...
0
votes
0answers
24 views

In which cases can SVM totally fail but Decision trees/Multilayer Perceptrons succeed in classification?

I have been working with a binary classification problem recently. The dataset contained 20 features. I used SVM with different parameter settings and got 50% accuracy (i.e. all predictions are ...
0
votes
0answers
29 views

Boosting - learners with weight?

I'm trying to understand the Boosting method and I'm very confused by what I see on the internet as there are different explanations and I'm not sure some of them are true. Here's what I've ...
0
votes
1answer
30 views

Writing the equation of boosted regression trees model [closed]

This is the formula for fitting a boosted regression trees from gbm package: ...
0
votes
0answers
14 views

Rule based ISLE Ensemble Generation

I come through a algorithm ISLE Ensemble Generation in machine learning. The following is the steps given in Elements of Statistical Learning: But I am unable to apprehend it and implement it in ...
1
vote
1answer
53 views

Using trees for conditioned linear regression

I want to explore using tree methods to condition a linear regression. Let's say the baseline regression is of the form $y = \beta x + \epsilon$. In addition, I have conditioning data, a vector $c \...
3
votes
1answer
113 views

Entropy Impurity, Gini Impurity, Information gain - differences?

I'm trying to understand the theory behind decision trees (CART) and especially the scikit-learn implementation. CART constructs binary trees using the feature and threshold that yield the ...
0
votes
0answers
13 views

Random Forest and Decision Trees as Variable Selection methods [duplicate]

I'm doing some variable selection task on a dataset with 130 columns(both categorical and numerical) and 15,000 rows. I read that random forest and decision tree can be applied for variable ...
1
vote
0answers
12 views

Classification and regression trees: what is a sensible value for the minimum number of observations in terminal nodes

Here I'll borrow some parameter names from the R rpart package for illustration. The most important hyperparameter for pruning a CART tree is the cost-complexity ...
0
votes
2answers
44 views

classification tree with rpart()

I'm using the rpart() to build a classification tree using R. I have no experience in this topic... Anyway, I started with the full model, and then I used the varImp() from "caret" to drop some ...
1
vote
0answers
18 views

Use GBM to find the optimum configuration of variables given certain conditions

I have a data set containing historical data of a SAG Ball Mill There are two types of variables: Mineral Characteristics, and Mill Paremeters. Mineral Characteristics are given and can not be ...
0
votes
0answers
7 views

Can regression trees or any other model be used to provide equation of separating hyperplane just like SVMs?

I am working on image segmentation. Suppose I have various spheres derived from the maximal ball algorithm. I want to generate a hyperplane that fits in the best way. Which methods other than SVM ...
1
vote
1answer
57 views

Decision tree vs linear SVM

Is it correct to say that, a difference between decision trees and linear SVMs is that the hyperplanes used by the decision trees are perpendicular to axis?
1
vote
1answer
30 views

CART Decision Tree Learning when some variables are ranges of continuous values

Could anyone kindly give some practical advice on how to deal with predictors of a range of values? For instance, I want to predict $Y$ based on features $X_{i}, \ i=1,2,...,N$. Some $X_{i}$s are of ...
1
vote
0answers
38 views

CART selection and *deselection* classification tree

I came across a line in Peterson, et al. (2016) that says: The specific settings applied in the rpart procedure ensured that only the largest subgroup would be ...
0
votes
0answers
36 views

Can't Validate Model Result

This is my first question and I am still new to data science and statistics, so please forgive me if this is a dumb or ill-posed question. I have two ensemble tree classifier models built on the same ...
1
vote
0answers
37 views

Performance of Conditional Inference regression Trees updating the influence function at each node

My goal is to compare the performance of $2$ models of trees using the Conditional Inference tree framework described in (ctree: Conditional Inference Trees), I am following the Partykit 2018. ...
12
votes
1answer
903 views

Depth of a decision tree

Since the decision tree algorithm split on an attribute at every step, the maximum depth of a decision tree is equal to the number of attributes of the data. Is this correct?
1
vote
0answers
16 views

CRAN Tree Package [closed]

I am using the tree package in R. What does (prune object)$k=-Inf mean? I mean what is the difference between k=0 and k=-Inf (k is the complexity parameter). ...