Questions tagged [cart]

'Classification And Regression Trees'. CART is a popular machine learning technique, and it forms the basis for techniques like random forests and common implementations of gradient boosting machines.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

Classifying single individual with ML with caret [closed]

After saving and loading a trained and tested algorithm (classifier), how can I classify (ill vs healthy) an unlabeled individual on the set of predictors used for training the model? I can not find ...
1
vote
0answers
9 views

How does decision tree decide which variable to use in next split?

The CART (or RPART) algorithm uses gini index to find a threshold value for a variable in each split. But how does it choose which variable will it use for splitting ?
0
votes
0answers
15 views

Which loss function is minimized in CART algorithm for binary classification?

Each ML algorithm is minimizing an objective function: when performing linear regression we minimize MSE, when performing logistic regression we minimize binary-cross entropy. This applies also to ...
0
votes
0answers
11 views

Decisions Tree - JOINT Probability rpart

Please describe if I could calculate joined probability for No class. I am using R and rpart. I want to check how probability will change if I make cuts under my tree like attached. 73%*34% + 71%*22%
0
votes
0answers
19 views

Creating a regression tree with a specific number of terminal nodes that minimize RSS

I am currently learning tree, but I have some issues understanding it. I would like to draw a tree with 4 terminal nodes for 5 observations that minimize the RSS. How could I do that manually or ...
1
vote
1answer
14 views

Interpreting a classification tree

I am currently trying to understand how I should interpret my classification tree. This is mine : How can I know which group most likely to swipe left, the probability of swiping left and the ...
1
vote
0answers
17 views

What factors will make a decision tree superior to a logistics regression model, taking into account the following? [closed]

Assume that the lift chart indicates that the decision tree is inferior, the C statistic is superior for the logistics regression also. What other factors will make a decision tree better, even in ...
0
votes
0answers
15 views

Decision tree with weights trained using RandomizedSearchCV - do I have to refit?

I trained a decision tree with weights using RandomizedSearchCV: ...
0
votes
0answers
7 views

Should I prune the tree trained on parameters obtained by Bayesian Optimization?

If the parameters for a decision tree are obtained by bayesian optimization, where the 10-fold cross validation error is minimized (or 1- error maximized) and afterwards the model is trained on the ...
0
votes
0answers
25 views

Should I use stratify parameter for scikit-learn train_test_split while dealing with highly unbalanced dataset?

I have a dataset with over 200000 records. Only 400 of are positive, which makes the data highly unbalanced. I cannot collect more data. At first I trained a decision tree. I used StratifiedKFold and ...
1
vote
0answers
57 views

Can someone explain Ripley's proof (1996) of the splitting of categorical variables?

Specifically I am referring to this theorem: 1) Suppose there are two classes. For a categorical feature , order the levels in increasing $p(1\mid x = x_i)$. Then a split of the form $\{x_1, ... x_{\...
0
votes
1answer
16 views

How can we include hourly traffic series data in the rows of train data set for training?

I have a classification problem where I am planning to use hourly traffic data for a day. Is there any way to compress it? instead of creating 24 predictors which account for hourly traffic?
1
vote
2answers
42 views

Does it make sense to use bayesian optimization for tuning of hyperparameters of decision tree model?

Does it make sense to use bayesian optimization for tuning of hyperparameters of decision tree model? I have not found any article or anything related to this, as BO is usually used for black-box ...
0
votes
0answers
21 views

Does random forest (and, decision tree) require an independent observation assumption? [duplicate]

I am wondering if random forest models require an independent observation assumption. My date includes observations from the same participants, but I do not have a way to identify each participant. ...
4
votes
0answers
23 views

How is EMSE derived for causal trees in Athey and Imbens (PNAS 2016)?

Athey and Imbens build a non-parametric matching procedure to identify and estimate causal effects. To this end, they minimize the expected mean squared error (EMSE) of their procedure, but I don't ...
1
vote
1answer
18 views

Experimental data segmentation using trees based on means - could single trial estimates improve reliability?

I have some categorical data and measures of participants accuracy. Let's say that it is a quiz and we have 8 different categories: History, Geography, Physics and so forth. Each participant is ...
0
votes
0answers
5 views

Prevalence outcome with conditional trees

I'm creating a Conditional Tree model to predict the point prevalence of a certain event estimated in some health centers. I thought of using the proportional prevalence [0,1] as the outcome, but the ...
0
votes
1answer
49 views

Can a decision Tree split on a question with two features?

If I have two classes that are cleanly separated by a single diagonal line as shown below, can I have a depth 1 decision tree using both features on the question x_2 > x_1 or can we not use two ...
0
votes
0answers
36 views

Use of Random Forest in a paper

I am currently reading a paper from a Geophysics journal in which the authors apply a random forest to data sets from shear laboratory experiments. I am new to machine learning, and I'm confused about ...
0
votes
0answers
8 views

How to interpret 'estimated rate' in recursive pationing analysis used for survival analysis in R /rpart?

I am using rpart for a survival analysis. I am not sure how to interpret the value labeled 'estimated rate' in a RPA used for survival analysis. I have ...
0
votes
0answers
29 views

randomForest prediction for zero-size nodes

I have just realized that despite the documentation ...
0
votes
1answer
21 views

Prediction for Regression trees

I know how to do prediction for classification trees, however I've never covered regression in class. What measures can you use as a prediction score,and how do you do it in R? I've only done this so ...
1
vote
0answers
45 views

Random forest [R]: why is my OOB RMSE so much smaller than test RMSE?

I'm doing the kaggle challenge on timetravel predictions where the task is to predict the duration (Y) of a uber trip given some information about the start and end coordinates and the time the trip ...
0
votes
0answers
5 views

repeatedly select the character in the regression decision tree using least squares method

We know that for the given training data $$D = \{(x_1, y_1),\cdots,(x_n, y_n)\}$$ here $x_i = (x_i^1,x_i^2,\cdots,x_i^m),$ to build a regression decision tree, ...
0
votes
1answer
22 views

How entropy is calculated when a non categorical feature is available when using Decision tree or random forest algorithms?

How an entropy is calculated on non-categorical feature containing big amount of unique numbers? Let me give you an example: When we're having a categorical ...
0
votes
0answers
9 views

Symmetrical distributed variable in multi-nominal classification ML model

I was trying to build a multi-nomial model on a 5-level dependent variable. There is one variable X that I really want to include in model, but show no importance in final output. It is a (0,1) ...
1
vote
0answers
12 views

How feature is selected to be a decision node after splitting the first root node using entropy in decisions trees?

In this article on how entropy is calculated and how the decision tree is built, the writer chose to split the sunny branch into ...
0
votes
0answers
11 views

Multilevel Classification Trees

I have a medical dataset with three levels: individuals, tested repeatedly over time, in three different settings. I would like to do some CART analysis for inference and prediction, i.e. to explore ...
1
vote
0answers
18 views

Can we extend the XGBoost algorithm to higher order steps?

In XGBoost, the idea is at every round of boosting we add an additional model (a decision tree in XGBoost for trees). This model is learned to optimize the second order Taylor expansion of the loss of ...
1
vote
1answer
24 views

Decision trees - hypothesis test for quality of split

I was just introduced to the concept of decision tree. I read that hypothesis testing can be used to asses the quality of each split $$H_0: \text{split was bad}$$ $$H_a: \text{split was good}$$ I ...
1
vote
1answer
17 views

Help understanding CART trees notation

I was reading the Decision Trees user guide of sklearn to understand some of the underlying mathematics behind trees. Everything was fine until I stumbled upon some notation I'm not understanding. ...
1
vote
3answers
55 views

Accuracy, Sensitivity, Specificity, & ROC AUC [duplicate]

In the context of predictive modeling, when comparing clasification models, What statistic should be considered more important over the others: Accuracy, sensitivity, specificity, or area under ROC ...
0
votes
0answers
15 views

Alternative algorithms for training CART models and decision trees

I am studying alternative algorithms for training decision trees. I am well aware of "standard" algorithms like CART, C4.5, ID3, and so on. I would like to find different algorithms, maybe based on ...
0
votes
0answers
20 views

Decision tree - least square empirical improvement

I'm looking for an explanation of formula (35) of the Gradient Boosting paper of Friedman [Friedman 2001, Greedy function approximation: a gradient boosting machine]. Here the least-squares ...
0
votes
1answer
16 views

Alternatives to 1SE Rule for Validation Set Parameter Tuning

I have a general question regarding parameter tuning on a held-out validation set (read: NOT cross validation, but a single held-out set of data). Suppose I would like to tune a parameter in a ...
3
votes
1answer
131 views

Regression tree splitting (CART, Scikit Learn) [closed]

I am working with a Random Forest using scikit-learn and still have some questions and thoughts I am not sure about regarding the splits in regression trees: 1. It seems to me that scikit-learn's ...
1
vote
0answers
25 views

How to deal with features that are only possible for some sample in Tree based algorithms?

If I have a data set where, for instance, I am predicting something about Bank customers. Some customers have mortgages and some don't. For those with mortgages I want to include how long until ...
0
votes
0answers
27 views

Isolation Forest Numerical Example

I'm looking for a proper numerical example to understand Isolation Forests Algorithm correctly. I've read the paper : https://github.com/mgckind/iso_forest/blob/master/icdm08b.pdf, but I want to ...
1
vote
0answers
14 views

Trying to improve the generalization ability of decision tree with bagging or random forest

We are fitting a regression tree on a sample of about 5000 observations with 5 predictors. The data stems from 2 sources. There are reasons to suspect that the tree will not have exaggerated ...
1
vote
0answers
34 views

How to construct a Probability Mass Function from a sample of data

In decision tree learning, specifically when calculating Gini impurity, I understand that probabilities are assigned to a class label based on the node label proportions, such as, ...
0
votes
0answers
9 views

Can we use joint entropy to split my nodes in a multi-objective decision tree?

Since for single objective decision tree (CART model), we can use information gain or difference in entropy (other than GINI index, Chi-Square) to split the nodes by choosing the attribute with ...
3
votes
1answer
101 views

Isolation Forest and average/expected depth formula

The Isolation Forest algorithm (Liu, Fei Tony, Kai Ming Ting, and Zhi-Hua Zhou. "Isolation forest." 2008 Eighth IEEE International Conference on Data Mining. IEEE, 2008. - link: https://cs.nju.edu.cn/...
4
votes
3answers
184 views

Feature Importance for Linear Regression

Is there a way to find feature importance of linear regression similar to tree algorithms, or even some parameter which is indicative? I am aware that the coefficients don't necessarily give us the ...
0
votes
0answers
12 views

Conditional Regression Trees

I was wondering how to properly use conditional regression trees and interpret them [ctree() in R]? For instance, if I have the following model: Y ~ x + b + c + x:c + b:c + x:b and it shows that x ...
1
vote
1answer
40 views

Decision Trees - how does split for categorical features happen?

A decision tree, while performing recursive binary splitting, selects an independent variable (say $X_j$) and a threshold (say $t$) such that the predictor space is split into regions {$X|X_j < t$} ...
3
votes
1answer
31 views

can we use any learners in gradient boosting instead of trees?

As we are simply trying to predict residuals from weak learners and aggregating them, can we use any weak learners in gradient boosting machines instead of trees ? If so, why are the all the gbm ...
1
vote
1answer
104 views

Formula for E(MSE) from Recursive Partitioning for Heterogeneous Causal Effects (Athey and Imben 2015)

I'm reading Recursive Partitioning for Heterogeneous Causal Effects (Athey and Imben 2015) and I'm confused about the formula for $\hat E[\mu(x;\Pi)]$ on page 8. The formula offered in the paper is ...
0
votes
1answer
32 views

Manually adjusting forecasting model bias

I am trying to build an efficient forecasting model to predict sales in the future. I managed to obtain a first pretty solid model using a LSTM network. However, it wasn't sensible enough to large ...
2
votes
1answer
26 views

Scikit's permuted features in decision tree implementation

In the Scikit's docummentation of decision trees I found a note: "The features are always randomly permuted at each split. Therefore, the best found split may vary, even with the same training data ...
0
votes
0answers
23 views

Balancing the training AND the test sets

I want to understand the relationship between a target variable (e.g. the user clicked a button) and a set of predictive variables (e.g. the user has spent x minutes on the page, the button is blue, ...