Questions tagged [cart]
'Classification And Regression Trees'. CART is a popular data mining technique.
2
votes
0answers
14 views
In Gradient Boosting Tree, why do we fit the tree on the residuals and not on the sum of the previous function and the residuals?
In the Gradient Boosting Tree algorithm, as described in https://en.wikipedia.org/wiki/Gradient_boosting#Gradient_tree_boosting, we update the previous model $F_m$ by adding the results $h_m$ of the ...
3
votes
1answer
88 views
Please correct my assumption on how regression trees work
I'm trying to understand how regression trees work, I've been experimenting with catboost and xgboost in python, and I'm getting results which I don't expect, can someone please clarify (and apologies ...
0
votes
2answers
53 views
Why are decision trees considered supervised learning?
It seems to work similar to clustering algorithms, where data does not have to be labeled, and the algorithm creates it's own labels/groups based on feature similarities...
0
votes
0answers
11 views
How to reduce the number of rules in decision tree with Support and Confidence
The following is a set of rules in decison tree. How to reduce the number of rules in the set with Support and Confidence?
If Ascites = 'Yes' then if Class = 'Live' then if Spiders = 'Yes' then if ...
1
vote
0answers
10 views
Rpart and subsets
I am getting unexpected results from an rpart model, where the model selects two variables, one of which is a subset of the other. This in itself is not unexpected, but the seemingly odd thing is that ...
0
votes
1answer
28 views
How do I use a random forest regression once trained [closed]
Similar questions have been asked on these two posts: random forest how to use the results and How to use random forest for regression after it is trained but I feel like the replies didn't give ...
0
votes
0answers
16 views
How to prevent GBDT from splitting on uninformative features?
I'm looking into using feature importance scores from GBDT for feature selection. Although GBDT does not need manual feature selection, the number of features is a restriction of the production system ...
1
vote
0answers
29 views
Model selection using p-values - tree inference
Suppose I have some i.i.d. normal observations from $\mathbb{R}^f$ with parameters $(\mu, \Sigma)$ and $\Sigma$ is known to be the identity matrix.
I have the following hypotheses:
$H_0^i$: $\mu_i = ...
0
votes
0answers
4 views
Few Queries regarding CART and logistic algorithms and missing data
I have few queries regarding ML algorithms and data, It would be great if you can provide some feedback on that.
Which is best package to impute the missing data (currently using MICE in R) and how ...
1
vote
1answer
96 views
Using a decision tree to predict a relevant location to a user
I am trying to create a decision tree to predict new locations a user would like to visit based on previous locations they have liked.
Here is my problem, I have two data sets,
a user data set ...
0
votes
0answers
42 views
Theoretical bound on the uncertainty provided by decision trees?
Decision trees have the property that they provide both a prediction and a probability for this prediction (scikit-learn's predict_proba method; Section 3.4 of Data ...
1
vote
0answers
15 views
Why do RMSE values increase on a smaller tree (RPART)
AIM: I want to understand why does RMSE increase on a smaller tree.
CONTEXT: I am learning the rpart algorithm. I had some data,...
0
votes
0answers
10 views
Using cosine of measurement time as feature for decision trees VS NNs
I have a regression data set and I'm trying to do some feature engineering.
The data set is foot fall coming into a store measured on the hour.
I'd like to include the time of measurement as a ...
0
votes
0answers
14 views
How to do a right model selection for modeling decision trees in R
I struggle with modeling a decision tree. I am trying to model both types (classification and regression), but let's stay for the first with regression tree.
I do have a large data set of 200000 ...
11
votes
4answers
2k views
Random Forest and Decision Tree Algorithm
A random forest is a collection of decision trees following the bagging concept. When we move from one decision tree to the next decision tree then how does the information learned by last decision ...
0
votes
0answers
7 views
How to extract and filter rules from C5.0 package for making fuzzy rules
I have project on a ML dataset. I have performed decision tree using C5.0 package in R.
For the next part I want to make fuzzy logic based on C5.0 results. I have 2 questions:
1- How can I filter and ...
0
votes
0answers
28 views
Interpreting printcp for classification trees in R
I'm trying to fit a classification tree model to the credit card fraud data from kaggle using rpart in R.
Once I come up with a model and call printcp(), I get the following output:
...
0
votes
0answers
5 views
Method for predicting whether a change in survey presentation had significant effect on survey response
I am not experienced in qualitative research in general and so I am hoping to get some guidance on this particular question I am dealing with (and perhaps from there I can extrapolate to some more ...
0
votes
0answers
5 views
What does the attribute splitter in DecisionTreeClassifier do?
The splitter attribute in the constructor of DecisionTreeClassifier(or DecisionTreeRegressor) can take two values 'best' or 'random'. What does the attribute splitter in DecisionTreeClassifier do?
2
votes
1answer
59 views
Gradient Boosted Trees vs Neural Network for limited data [closed]
I have a classification problem, with about 10 different inputs, some boolean, some categorical (and unrelated to each other), some being a float between 0 and 1, which need to be mapped to 4 ...
2
votes
1answer
64 views
Regression tree with nested data repeated in time (GLMERTREE, REEMTREE or REEMCTREE)
I work on the predation of seeds by insects (carabidae), and I am particularly interested in the effect of community composition on predation. I would like to know if the best predation rates are ...
1
vote
0answers
24 views
Boosted decision trees: in which situations are “deep” decision trees performing better?
The general idea of boosted decision trees is to use very simple trees in the following manner (simplified, for intuition only):
start with a simple tree,
fit another simple tree on the residuals,
...
0
votes
0answers
33 views
Penalty matrix on Decision Tree
I am doing a project on predicting credit risk and am trying to use decision trees. First, I trained the tree without any penalty matrix.
Next, I defined my penalty matrix as
$$
\begin{matrix}
0 &...
0
votes
0answers
8 views
Data synthesis approach for sample sample prediction
I have to forecast sales of a temporal SKU which only sold one month in a year. I have sales figures for nearly 1000 different sales locations.
Moreover, my data is limited for only 3 years. So I ...
0
votes
0answers
15 views
Will a regression tree work for grouping combinations of variables?
I would like to discover patterns in my dataset. For example,
Var1 var2 var3 var4 var5 var6
1 0 0 0 0 0
0 1 0 0 0 0
0 1 1 0 0 0
I have decided to use ...
2
votes
2answers
37 views
Depth of Decision Tree
If there are only categorical variables in the dataset, will the depth of the decision tree be equal to the number of attributes? If not, can a value be split again?
1
vote
0answers
12 views
Is it possible to use Information Gain metric for CART?
I've been looking for an example of CART using Information Gain but haven't found one. This made me wonder if it was even possible, so I tried to train one manually (by hand) using the dataset below ...
0
votes
1answer
20 views
DECISION TREE : How to calculated for repeat decision noded such as this picture (C5.0 Algorithm -Decision tree)
I confused about decision tree such as this picture why repeat decision node.Could you please explain that decision tree. thank you
2
votes
2answers
29 views
Decision Tree in layman terms
How can we describe decision tree in laymen's language and what are the major fields that require this?
0
votes
1answer
25 views
Question when using sklearn's DecisionTreeClassifier
sklearn's DecisionTreeClassifier is not behaving as I expected. From the following:
...
2
votes
0answers
25 views
Hypothesis space of regression/classification tree
I'm wondering how to exactly define the hypothesis space of a tree.
For simplicity start with a regression tree
$f(x) = \sum_{m=1}^Mc_m\mathbb{I}(x \in R_m)$
Where $c_m$ are the ( $M$ ) constant ...
0
votes
1answer
36 views
`randomForest` predict() for continuous variable: unexpected output?
I am trying to understand how predict() in randomForest() in R computes the predicted values for a continuous y? My ...
10
votes
1answer
294 views
Boosting AND Bagging Trees (XGBoost, LightGBM)
There are many blog posts, YouTube videos, etc. about the ideas of bagging or boosting trees. My general understanding is that the pseudo code for each is:
Bagging:
Take N random samples of x% of ...
0
votes
1answer
19 views
How to categorize data as others if training set is not available?
I run into a problem. I am using the decision tree to classify the incident category based on the short description the user has used while logging the ticket.
I have the training data only for 5 ...
0
votes
0answers
13 views
Why is the sqrt(n_features) the default maximum number of features for the best split in RandomForestClassifier? [duplicate]
Why does sklearn.ensemble.RandomForestClassifier references have $\sqrt{n}$ in the max_features implementation and why does randomForest in R seem to have the same $\sqrt{n}$ default? I am looking for ...
1
vote
2answers
66 views
multicollinearity resulting in high variance
Section 8.7.1 of Elements of Statistical Learning talks about high variance in a classification tree due to high correlation between features. What is the intuition behind this? I would think that ...
0
votes
0answers
18 views
How can I run a decision tree algorithm with a specific hierarchy of variables and with many missing values?
I asked students in learning groups what their biggest learning problem was "today" for each learner. The biggest problem could either be "motivational" (=motivation problem) or cognitive (="knowledge ...
0
votes
1answer
38 views
Classification Tree or Regression Tree?
I have time series data: students that learned in groups for minimum 3 times and maximum 10 times and for each learning group session had to state if they faced a motivational OR a cognitive problem, ...
1
vote
1answer
122 views
optimal decision tree np-hard
Reading Elements of Statistical Learning and it says that decision trees are often constructed using greedy algorithms because it is computationally infeasible to create an optimal decision tree. ...
0
votes
1answer
49 views
Decision tree without the “tree”
I would like to construct something like a decision tree. However, instead of using "recursive partitioning" to build a tree, I would like to find an optimal set of "global" splits.
For example, in a ...
0
votes
1answer
24 views
Merge one label with one information for classification problem or multi-label classification
I want to build a model to support decision making in order to propose or not loan insurance to clients. Because sometimes clients asking loan and loan insurance have less chance to have their loan ...
2
votes
1answer
19 views
what if a decision tree does not result in leaves with one class each?
A decision tree can result in leaf nodes that have samples from multiple classes. Is the algorithm at that point to simply vote on the class?
0
votes
0answers
10 views
Is pseudo-r2 enough to validate a CART?
I run some Classification and Regression Trees (CARTs) and computed the pseudo-$R^2$ from McFadden. Is that enough to validate the trees or do I need some other test to be sure there is no overfitting?...
0
votes
0answers
53 views
Difference between Random Forest and Random Subspaces/Patches
When fitting a Random Forest model, a subset of the features is randomly considered at the splitting of each node. E.g., if $p$ is the number of features, then at each node in each tree, $\sqrt{p}$ ...
0
votes
0answers
33 views
Which algorithms for mixed type datasets (binary classification)?
I am new to machine learning and I am trying to implement a model for a binary classification problem (output class 0 or class 1), and wondering which algorithms I should consider, since my dataset is ...
1
vote
0answers
33 views
What is the relation between minimum instances per node and max depth?
In bagging and boosting models like random forest and xgboost we have hyper-parameters like minimum instances per node and max depth. If max depth is high the minimum instances per node will be less ...
3
votes
1answer
51 views
Which model for feature importances?
When wanting to find which features are the most important in a dataset, most people use a linear model - in most cases an L1 regularized one (i.e. Lasso).
However, tree based algorithms have their ...
2
votes
0answers
22 views
True or False? Not testing data is needed for CART if there is not future prediction
I am analysing a data set with a relatively a small sample size. For the nature of my data, it is considered already quite large, but for its statistical power it is just in the lower limit. I sampled ...
0
votes
0answers
12 views
Small sample size for partitioning and evaluating a CART
I have a data set of 130 samples. If I partition the data 70/30 % to run a conditional tree, CART, and then evaluate it, the results are different than if I run the 100% of my data in the CART. When I ...
0
votes
0answers
43 views
Decision Tree with unbalanced dataset in SAS
I have a dataset with a binary target variable. This variable is highly imbalanced i.e. the # of True case is ~1% and # of False cases is ~99% The other limitation I have is that I can only use ...
