Skip to main content

Questions tagged [cart]

'Classification And Regression Trees', also sometimes called 'decision trees'. CART is a popular machine learning technique, and it forms the basis for techniques like random forests and common implementations of gradient boosting machines.

Filter by
Sorted by
Tagged with
1 vote
1 answer
15 views

Interpretation of a decision tree plot

For a paper, I am training different models and using LIME to simplify the blackbox models into a transparent decision tree model that I can visualize with view(tree, "mode", "graph&...
1 vote
1 answer
228 views

Decision tree when leaf nodes are numerical values

I'm having a dataset, where I created a decision tree. My output variables are binary values. I selected two features and the decision tree was generated using R. This is my code. ...
2 votes
1 answer
231 views

Does adding new categorical data decrease prediction performance in classification?

I have a dataset in which new data comes in everyday. There are categorical variables in the inputs. As a result, I use one-hot-encoder to create a dummy variable. If a new categorical comes in, the ...
0 votes
1 answer
961 views

GridsearchCV() gives optimum criterion for Decision Tree should be entropy, but why am I getting better accuracy with Gini?

I ran this code ...
3 votes
3 answers
2k views

Evaluating mean-squared error

I am running a Regression Tree experiment where I am using Mean Squared error to test my regression trees. I am getting a large Mean Squared Error (MSE) but I don't know how to evaluate if it is too ...
2 votes
2 answers
63 views

Is feature importance given by decision tree universal?

I'm wondering that if I have a set of features on a fitted classification decision tree with relative low feature importance, would it mean that these features would also be negligible when fitted ...
2 votes
1 answer
2k views

How to estimate the leafsize of the kd-tree?

The kd-tree implementation proposed by the scipy python libray asks for the value of the leafsize parameter that is to say the maximum number of points a node can hold. It is by default set to 10. ...
0 votes
0 answers
10 views

What time window to use for independent variables?

I have 10000 listings on various days. My y variable is % of transactions that happened 7 days before listing with a loss > 0 is 30% or more then 1 else 0. Now losses can happen before listing also....
4 votes
1 answer
2k views

Random forest splitting rule and importance in package ranger

The package ranger implements random forests in R. Among other things, the function used to fit a random forest allows to choose among several splitting rules, and ...
2 votes
0 answers
352 views

How is Gini impurity related to accuracy when predicting the majority class?

For simplicity, consider the binary case, where we have a set of elements with each element belonging to one of two classes (0 or 1). Let p(j) be the proportion of ...
1 vote
1 answer
306 views

Which kind of distributions can decision trees learn (well)?

Suppose I have a classification task in $\mathbb{R}^d$, given by a distribution $P$ and training data $D$. The decision boundary is $$B = \{x \; : \; P(Y=1|x) = P(Y=0|x)\}$$ Assume I am learning a ...
1 vote
1 answer
703 views

How is the threshold parameter practically selected for Scikit learn's decision tree algorithm and how to determine depth of tree?

I am referring to the so-called optimized CART algorithm that is explained on Scikit learn's website: https://scikit-learn.org/stable/modules/tree.html#mathematical-formulation I would appreciate if ...
1 vote
0 answers
26 views

XGB predict_proba estimates don't match sum of leaves [closed]

When using an XGB model in the context of binary classification, I observed that the test estimates given by predict_proba were close but not equal to the results I ...
2 votes
2 answers
842 views

Ctree in R: how optimal is the optimal split point?

Hi I’m fairly new to using decion trees. I understand that to find the best split points, the ctree algorithm maximises a certain test statistic. I am interested to inspect the values of the test ...
3 votes
1 answer
297 views

Complexity associated with decision trees

According to the sklearn documentation on decision trees: The cost of using the tree (i.e., predicting data) is logarithmic in the number of data points used to train the tree. Could somebody ...
0 votes
0 answers
17 views

How to calculate the statistic for ctree function?

...
73 votes
3 answers
172k views

How to actually plot a sample tree from randomForest::getTree()? [closed]

Anyone got library or code suggestions on how to actually plot a couple of sample trees from: getTree(rfobj, k, labelVar=TRUE) (Yes I know you're not supposed ...
0 votes
1 answer
19 views

Standardise features for a tree based model or logistic regression

I am trying to understand if one should standardise features for all models and when does it make sense to do so. Is the below statement true? If yes, could you give a bit of explanation please. ...
0 votes
1 answer
18 views

Understanding rpart Tree Splits: Why Did a Node Remain Terminal Despite Potential Split?

Setup Consider two trees grown with rpart() from the rpart R package, in which the only thing that changes is the minbucket ...
0 votes
0 answers
16 views

Missing features in decision tree based algorithms

I have a medium-sized dataset consisting of many features, some of which can contain missing values. I want to predict a variable using an algorithm that employs decision trees (specifically XGBoost, ...
2 votes
1 answer
238 views

Can you explain this description of tree pruning in Intro to Statistical Learning?

The underlined sentences below from p. 331 in An Introduction to Statistical Learning have me scratching my head: Given that the splitting algorithm always finds the best next split in terms of error ...
0 votes
0 answers
19 views

Dataset for impact prediction in a container

I have a sensor that captures when there was an impact in a container. With this i have a dataset that contains data of when the device communicated and there was no impact and when the device ...
1 vote
2 answers
1k views

How do I optimize decision tree regression algorithm implemented in R?

I'm only getting an accuracy of 59% using the following implementation calculated using the diag(sum(cm)) and sum(cm) functions. ...
0 votes
0 answers
9 views

What are the problem with running binary classification on panel data using logistic regression?

I am a newbie and I am trying to understand why one cannot run algorithms like logistic regression or decision trees on panel dataset ? I am running an employee attrition model where the dependent ...
3 votes
1 answer
45 views

Features available during training but not at prediction

Broadly, my motivation is to understand if/how features available during training but not at prediction can be used to improve the prediction accuracy of a machine learning model. This question is ...
2 votes
1 answer
2k views

Partitioning with cross validation?

I am new to data analytics having only started exploring the field this week. I have downloaded KNIME and am working with a single dataset to try out different classification algorithms. I am ...
2 votes
1 answer
388 views

Difference between One Rule Classifier and Decision Stump in WEKA

WEKA Explorer seems to come up with two different models for OneR (rules) and Decision stump (trees). Is has to be the underlying measure of "best split" that is different. But for a single ...
0 votes
1 answer
42 views

How does Cross Validation work in decision trees (or tree ensembles)

I've been working with tree-based models for a long time and I never really asked myself how cross-validation would work when building a tree. For the sake of this question, suppose I've split my ...
2 votes
1 answer
584 views

Using Priors in Decision trees

I have been building a predictive model using Decision Trees. The data is highly skewed (only 2% of the target variable is "Yes" and others are "No") and I have to increase the precision for "Yes". I ...
1 vote
0 answers
82 views

Why does feature importance decrease for highly correlated variables?

I am investigating the relationship between correlation between features and its impact on their feature importances using sklearn's DecisionTreeClassifier algorithm. I manipulated the correlation of ...
5 votes
1 answer
2k views

What's the relation between Canonical Correlation Analysis (CCA) and Regression?

I'm wondering if CCA is just a feature transformation method. Can I use it for predicting continuous variables like in regression methods? What I'm doing is to use CCA to transform my training and ...
0 votes
1 answer
39 views

How to handle correlated variables before using Recursive Feature Elimination?

I have seen a few Kaggle notebooks that list without reason that RFE works better when removing correlated variables. I struggle to see the reason why so I conducted some of my own research and would ...
0 votes
0 answers
20 views

Decision Tree: What do you do if all your classes have same frequency or you have no data points at the leaf node?

I was solving a question and I realized after splitting the tree on every attribute. I reached a point where if my final attribute value is "No" then all the classes have equal frequency. if ...
3 votes
1 answer
906 views

How are missing values exactly handled in C4.5 decision trees?

I quote Tom M. Mitchell's words on this topic: " A second, more complex procedure is to assign a probability to each of the possible values of A rather than simply assigning the most common value ...
0 votes
0 answers
11 views

Cubist Model Tree Overlapping Rules

Can someone help me understand the output of a cubist model created by the R Cubist package? The documentation package manual for objects of class "cubist" seems to be nonexistent. Looking ...
10 votes
1 answer
10k views

How to include interaction terms in R/tree model?

I have read at many places that tree is good for uncovering complex dependencies among predictor variables. From Tree models in R: The recursive structure of CART models is ideal for uncovering ...
0 votes
0 answers
39 views

Fitting probabilities for decision trees

I perform an accident analysis with decision trees. Some cases are clear (all info exists), other ones haven’t it. By my previous experience, I can add missing data (restoration costs). But I don’t ...
5 votes
1 answer
1k views

C4.5 How to select the split point (threshold) for a Continuous Attribute

Using the "play golf" or "play ball" data (listed at the bottom), to pick the root node we look at Outlook, Temperature, Humidity, and Wind, to see which has the highest GainRatio. Now, Outlook will ...
1 vote
1 answer
85 views

Isolation Forest Numerical Example

I'm looking for a proper numerical example to understand Isolation Forests Algorithm correctly. I've read the paper : https://github.com/mgckind/iso_forest/blob/master/icdm08b.pdf, but I want to ...
0 votes
1 answer
284 views

How can I generate a plot of the partitions in Isolation Forests

I have seen this plot is used to indicated how anomalies are isolated via partitioning in Isolation Forests. Is there a library to automatically plot this from a dataset? The plot I want to generate ...
0 votes
0 answers
13 views

Ensemble, merge or combine multiple lmertree objects

Working with the PISA data, which includes multiple achievement scores (plausible values) for each participant, I would like to run the same lmertree and ensemble ...
2 votes
2 answers
75 views

Why does my neural network consider different features important compared to my decision tree?

I built a neural network and a decision tree using very similar data sets (the only difference was the randomness of selecting the training vs testing set). The variables with the highest shapley ...
2 votes
2 answers
1k views

How does `rpart` in R decide on splits

library(rpart) library(rpart.plot) library(rattle) tree <- rpart(lm(Sepal.Length ~ Sepal.Width + Petal.Width, data = iris)) fancyRpartPlot((tree)) From my ...
1 vote
0 answers
39 views

How does random forest determine the possible split values to assess?

I understand the basics of how decision trees / random forests are built, but one thing I still don't have a good answer for is how the putative split values are identified. So suppose we have mtry = ...
0 votes
2 answers
65 views

Classification and regression trees splitting depth - how it works?

I am trying to understand how a CART tree grows, So I am growing a tree step by step, and I am finding a strange (?) behavior. Let me show this by means of an example: I will use the titanic data set ...
2 votes
2 answers
3k views

Which assumptions should be checked for regression tree to validated model?

I am working with regression tree. I have four predictors. There is a exponential relationship between predictor and dependent variable. But after building predictive model I cannot understand whether ...
24 votes
3 answers
8k views

What is the relationship between the GINI score and the log-likelihood ratio

I am studying classification and regression trees, and one of the measures for the split location is the GINI score. Now I am used to determining best split location when the log of the likelihood ...
0 votes
0 answers
40 views

Derivation of the formula for E(MSE) in Recursive Partitioning for Heterogeneous Causal Effects (Athey and Imben 2015)

This post answers a question from the Recursive Partitioning for Heterogeneous Causal Effects paper by Athey and Imbens. However, I am blocked at an even earlier stage from the previous post. I don't ...
2 votes
1 answer
156 views

Gini impurity greedily optimises a loss function in decision trees

I am trying to understand how the Gini criterion for decision decision tree construction actually greedily optimises a loss function. The Gini impurity, sometimes also called Gini index, for a region (...
10 votes
1 answer
2k views

Using an RMSE with derived confidence interval, to generate a prediction interval for an estimate

Previous questions have asked about creating prediction intervals for estimates derived from random forests or boosted regression trees, in a similar way to is easily achieved with linear regression ...

1
2 3 4 5
26