Skip to main content

Questions tagged [pruning]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
2 votes
1 answer

Can you explain this description of tree pruning in Intro to Statistical Learning?

The underlined sentences below from p. 331 in An Introduction to Statistical Learning have me scratching my head: Given that the splitting algorithm always finds the best next split in terms of error ...
Jon's user avatar
  • 181
0 votes
0 answers

Cost complexity pruning in random forests

When choosing the optimal alpha for cost complexity pruning in a single decision tree, we can directly look at the subset of effective alphas. However, in the context of random forests, there isn't an ...
user_12345's user avatar
0 votes
0 answers

Decision trees: Measure of split quality which takes into account rare values

I am working on a classification problem in which the positive class is very rare. The dataset consists of categorical variables, as shown in the example below. The variables are hierarchical, in the ...
bjarkemoensted's user avatar
2 votes
1 answer

partykit::ctree(): Pruning a binary-classification tree based on practical relevance instead of statistical significance

I'm creating a binary classification model to develop relevant segments for a business problem. The ctree-function does a great job, especially in combination with the ...
stats-hb's user avatar
  • 289
2 votes
1 answer

Does pruning Deep Neural Networks at initialization even make sense?

I recently started exploring pruning methods for Deep Neural Networks and stumbled on some interesting papers suggesting algorithms for unstructured pruning at initialization (e.g. SNIP), i.e. ...
David's user avatar
  • 119
0 votes
0 answers

What happens to the accuracy of a decision tree after pruning?

What happens to the accuracy of a decision tree when it is pruned? Can be higher than the accuracy of the fully-grown decision tree?
san's user avatar
  • 11
1 vote
2 answers

How to handle highly correlated observations (rows)

What is the best practice to handle highly similar/ autocorrelated observations (rows) in a data set. These highly similar rows could come from recording (some of the) observations at too close ...
Ggjj11's user avatar
  • 1,373
0 votes
0 answers

Algorthm for discerning which top X values in a list are statistically ranked higher than the bottom Y

Given a ranked list, which contains values with high uncertainty, I would like to remove as many of the middle values that have high overlap in uncertainties as possible, and be left with more or less ...
Charlie Crown's user avatar
0 votes
0 answers

Alternative to post or pre-pruning

Usually, to better generalize and have a better understandability of the underlying model, we prune the decision trees. And in some cases, it is still large and difficult to interpret. Is there any ...
user27286's user avatar
  • 299
1 vote
1 answer

What is the difference between network sparsification and model pruning

What is the difference between network sparsification and model pruning? I watched USENIX ATC '21 - Octo: INT8 Training with Loss-aware Compensation and Backward Quantization for Tiny (at 01:29sec) ...
Mas A's user avatar
  • 203
1 vote
0 answers

How does cost_complexity_pruning_path in sklearn calculate effective alphas when pruning a decision tree?

I know when pruing DecisionTreeRegressor, we can leverage cost_complexity_pruning_path method to get a list of effective alphas. ...
Alice Wang's user avatar
-1 votes
1 answer

What is the difference between Regularization, optimization, and pruning?

Regularization Techniques in Deep Learning = reduces or solves overfitting problem. Optimizing Neural Network Structures with Keras-Tuner = reduces the connections and number of neurons for optimal ...
user366312's user avatar
  • 2,140
1 vote
0 answers

Cost complexity pruning decision trees

I am trying to understand cost complexity pruning in classification trees. I found that DecisionTree in sklearn has a function called cost_complexity_pruning_path, which gives the effective alphas of ...
Andreas Zaras's user avatar
0 votes
1 answer

Tree's branch ends to the same leaf twice?

I am in R using the DoctorVisits dataset from the AER package. I chose the column ...
Panagiotis Basiouras Serrano's user avatar
3 votes
1 answer

Overview of the main methods to prune decision trees

Could someone explain the main pruning techniques for decision trees. So something like the 3 most common techniques with a short explanation of how they work. I have looked online but this, ...
Trajan's user avatar
  • 449
3 votes
1 answer

Pruning in Decision Trees?

Pruning is a technique in machine learning and search algorithms that reduces the size of decision trees by removing sections of the tree that provide little power to classify instances. I know what ...
Thalassophile's user avatar
32 votes
3 answers

Why Not Prune Your Neural Network?

Han et al. (2015) used a method of iterative pruning to reduce their network to only 10% of its original size with no loss of accuracy by removing weights with very low values, since these changed ...
RoryHector's user avatar
2 votes
1 answer

Decision Trees: Cost Complexity Parameter and $-\infty$

I am reading the book titled "An Introduction to Statistical Learning with Applications in R" by James et al. On page 326, we perform cross-validation to determine the optimal level of tree complexity ...
Zachary's user avatar
  • 135
1 vote
1 answer

Neural network: finding irrelevant inputs

I am training a neural network and I suspect that some of the inputs might be irrelevant -- they might not have any relationship to the output. How do I identify these inputs so I can get rid of them? ...
Jessica's user avatar
  • 1,231
7 votes
2 answers

Neural network's weight reduction

Are there any algorithms/methods for taking a trained model and reducing its number of weights with as little negative effect as possible to its final performance? Say I have a very big (too big) ...
Mark.F's user avatar
  • 743
2 votes
0 answers

Determining alpha for pruning trees with cross-validation

following the answer from of Steffen to the question below: How to choose $\alpha$ in cost-complexity pruning? and slide 10 in: I'm still ...
user541057's user avatar
2 votes
1 answer

Cost complexity pruning and prediction error

I am reading the book titled "An Introduction to Statistical Learning" by James et al. There it is mentioned on page 309 that we pick the cost complexity parameter α to minimize the average Mean ...
Anup's user avatar
  • 135
2 votes
1 answer

Post-Pruning in partykit: the size of mob() tree

I am trying to build a multiple regression model while partitioning my data into subgroups based on additional set of covariates. While I implemented lmtree() or mob() in the "partykit" package, I ...
sunmee's user avatar
  • 23
5 votes
1 answer

How to obtain regularization parameter when pruning decision trees?

I'm having trouble understanding exactly how to obtain the regularization parameter when pruning a decision tree with the minimal cost complexity approach. Assume the cost complexity function is ...
Jacob H's user avatar
  • 922
4 votes
0 answers

Machine Learning and Flow Maximization

Has anyone ever seen machine learning (ML) used to assist a Max Flow algorithm? I have a very large directed graph that has some fractal characteristics, meaning that this large graph can be roughly ...
rafbrl's user avatar
  • 41
16 votes
2 answers

How to choose $\alpha$ in cost-complexity pruning?

In the following lectures Tree Methods, they describe a tree algorithm for cost complexity pruning on page 21. It says we apply cost complexity pruning to the large tree in order to obtain a sequence ...
itzjustricky's user avatar
2 votes
0 answers

pruning : why if T1 and T2 (2 subtrees) with the same risk imply that one must be a subree of the other

I don't understand the following assertion from "An Introduction to Recursive Partitioning" page 13. If T1 and T2 are sub trees of T with Rα(T1) = Rα(T2), then either T1 is a sub tree of T2 or ...
André Mayers's user avatar
3 votes
1 answer

Avoiding overfitting with linear regression trees

I use regression trees (R package rpart) in my statistical analysis, and have received a critical comment that this method amounts to a "hunting expedition" that will always produce a result ("...
robert's user avatar
  • 746