Skip to main content

Questions tagged [pruning]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
2 votes
1 answer
240 views

Can you explain this description of tree pruning in Intro to Statistical Learning?

The underlined sentences below from p. 331 in An Introduction to Statistical Learning have me scratching my head: Given that the splitting algorithm always finds the best next split in terms of error ...
Jon's user avatar
  • 181
0 votes
0 answers
94 views

Cost complexity pruning in random forests

When choosing the optimal alpha for cost complexity pruning in a single decision tree, we can directly look at the subset of effective alphas. However, in the context of random forests, there isn't an ...
user_12345's user avatar
0 votes
0 answers
15 views

Decision trees: Measure of split quality which takes into account rare values

I am working on a classification problem in which the positive class is very rare. The dataset consists of categorical variables, as shown in the example below. The variables are hierarchical, in the ...
bjarkemoensted's user avatar
2 votes
1 answer
141 views

partykit::ctree(): Pruning a binary-classification tree based on practical relevance instead of statistical significance

I'm creating a binary classification model to develop relevant segments for a business problem. The ctree-function does a great job, especially in combination with the ...
stats-hb's user avatar
  • 289
2 votes
1 answer
87 views

Does pruning Deep Neural Networks at initialization even make sense?

I recently started exploring pruning methods for Deep Neural Networks and stumbled on some interesting papers suggesting algorithms for unstructured pruning at initialization (e.g. SNIP), i.e. ...
David's user avatar
  • 119
0 votes
0 answers
142 views

What happens to the accuracy of a decision tree after pruning?

What happens to the accuracy of a decision tree when it is pruned? Can be higher than the accuracy of the fully-grown decision tree?
san's user avatar
  • 11
1 vote
2 answers
313 views

How to handle highly correlated observations (rows)

What is the best practice to handle highly similar/ autocorrelated observations (rows) in a data set. These highly similar rows could come from recording (some of the) observations at too close ...
Ggjj11's user avatar
  • 1,373
0 votes
0 answers
18 views

Algorthm for discerning which top X values in a list are statistically ranked higher than the bottom Y

Given a ranked list, which contains values with high uncertainty, I would like to remove as many of the middle values that have high overlap in uncertainties as possible, and be left with more or less ...
Charlie Crown's user avatar
0 votes
0 answers
69 views

Alternative to post or pre-pruning

Usually, to better generalize and have a better understandability of the underlying model, we prune the decision trees. And in some cases, it is still large and difficult to interpret. Is there any ...
user27286's user avatar
  • 299
1 vote
1 answer
628 views

What is the difference between network sparsification and model pruning

What is the difference between network sparsification and model pruning? I watched USENIX ATC '21 - Octo: INT8 Training with Loss-aware Compensation and Backward Quantization for Tiny (at 01:29sec) ...
Mas A's user avatar
  • 203
1 vote
0 answers
88 views

How does cost_complexity_pruning_path in sklearn calculate effective alphas when pruning a decision tree?

I know when pruing DecisionTreeRegressor, we can leverage cost_complexity_pruning_path method to get a list of effective alphas. ...
Alice Wang's user avatar
-1 votes
1 answer
2k views

What is the difference between Regularization, optimization, and pruning?

Regularization Techniques in Deep Learning = reduces or solves overfitting problem. Optimizing Neural Network Structures with Keras-Tuner = reduces the connections and number of neurons for optimal ...
user366312's user avatar
  • 2,140
1 vote
0 answers
365 views

Cost complexity pruning decision trees

I am trying to understand cost complexity pruning in classification trees. I found that DecisionTree in sklearn has a function called cost_complexity_pruning_path, which gives the effective alphas of ...
Andreas Zaras's user avatar
0 votes
1 answer
90 views

Tree's branch ends to the same leaf twice?

I am in R using the DoctorVisits dataset from the AER package. I chose the column ...
Panagiotis Basiouras Serrano's user avatar
3 votes
1 answer
564 views

Overview of the main methods to prune decision trees

Could someone explain the main pruning techniques for decision trees. So something like the 3 most common techniques with a short explanation of how they work. I have looked online but this, ...
Trajan's user avatar
  • 449
3 votes
1 answer
438 views

Pruning in Decision Trees?

Pruning is a technique in machine learning and search algorithms that reduces the size of decision trees by removing sections of the tree that provide little power to classify instances. I know what ...
Thalassophile's user avatar
32 votes
3 answers
5k views

Why Not Prune Your Neural Network?

Han et al. (2015) used a method of iterative pruning to reduce their network to only 10% of its original size with no loss of accuracy by removing weights with very low values, since these changed ...
RoryHector's user avatar
2 votes
1 answer
857 views

Decision Trees: Cost Complexity Parameter and $-\infty$

I am reading the book titled "An Introduction to Statistical Learning with Applications in R" by James et al. On page 326, we perform cross-validation to determine the optimal level of tree complexity ...
Zachary's user avatar
  • 135
1 vote
1 answer
175 views

Neural network: finding irrelevant inputs

I am training a neural network and I suspect that some of the inputs might be irrelevant -- they might not have any relationship to the output. How do I identify these inputs so I can get rid of them? ...
Jessica's user avatar
  • 1,231
7 votes
2 answers
1k views

Neural network's weight reduction

Are there any algorithms/methods for taking a trained model and reducing its number of weights with as little negative effect as possible to its final performance? Say I have a very big (too big) ...
Mark.F's user avatar
  • 743
2 votes
0 answers
2k views

Determining alpha for pruning trees with cross-validation

following the answer from of Steffen to the question below: How to choose $\alpha$ in cost-complexity pruning? and slide 10 in: https://web.stanford.edu/class/stats202/content/lec19.pdf I'm still ...
user541057's user avatar
2 votes
1 answer
588 views

Cost complexity pruning and prediction error

I am reading the book titled "An Introduction to Statistical Learning" by James et al. There it is mentioned on page 309 that we pick the cost complexity parameter α to minimize the average Mean ...
Anup's user avatar
  • 135
2 votes
1 answer
659 views

Post-Pruning in partykit: the size of mob() tree

I am trying to build a multiple regression model while partitioning my data into subgroups based on additional set of covariates. While I implemented lmtree() or mob() in the "partykit" package, I ...
sunmee's user avatar
  • 23
5 votes
1 answer
2k views

How to obtain regularization parameter when pruning decision trees?

I'm having trouble understanding exactly how to obtain the regularization parameter when pruning a decision tree with the minimal cost complexity approach. Assume the cost complexity function is ...
Jacob H's user avatar
  • 922
4 votes
0 answers
306 views

Machine Learning and Flow Maximization

Has anyone ever seen machine learning (ML) used to assist a Max Flow algorithm? I have a very large directed graph that has some fractal characteristics, meaning that this large graph can be roughly ...
rafbrl's user avatar
  • 41
16 votes
2 answers
20k views

How to choose $\alpha$ in cost-complexity pruning?

In the following lectures Tree Methods, they describe a tree algorithm for cost complexity pruning on page 21. It says we apply cost complexity pruning to the large tree in order to obtain a sequence ...
itzjustricky's user avatar
2 votes
0 answers
32 views

pruning : why if T1 and T2 (2 subtrees) with the same risk imply that one must be a subree of the other

I don't understand the following assertion from "An Introduction to Recursive Partitioning" page 13. If T1 and T2 are sub trees of T with Rα(T1) = Rα(T2), then either T1 is a sub tree of T2 or ...
André Mayers's user avatar
3 votes
1 answer
2k views

Avoiding overfitting with linear regression trees

I use regression trees (R package rpart) in my statistical analysis, and have received a critical comment that this method amounts to a "hunting expedition" that will always produce a result ("...
robert's user avatar
  • 746