The boosting tag has no wiki summary.
1
vote
1answer
35 views
The weight updating in adaboost
1.AdaBoost updates the weight of the sample By the current weak classifier in training each stage. Why doesn't it use the all of the previous weak classifiers to update the weight. (I had tested it ...
0
votes
0answers
27 views
How can I do logistic correction for boosting
Can anyone tell me if logistic correction is the best method to correct the probability of gradient boosting machine? If so, how can I do it?
1
vote
0answers
39 views
Which Regression methods are suitable for binary valued features and continuous output?
I want to build a machine learning model to regression on continuous output given binary valued features(0,1). the dimension of my problem is around 200. which of the flowing methods seems suitable ...
1
vote
4answers
127 views
Measuring representativeness of a sample using covariates
I was provided with quite a small sample of labeled (variable of interest) observations to train a model to predict unlabeled observations. All the observations are associated with many covariates. ...
0
votes
0answers
20 views
Obtaining final classification score using AdaBoost predict function
If I understand correctly, predict.ada() returns an $n$ by 2 matrix of class probabilities for each classifier used in $n$ iterations. How can I obtain the final classification on scale of [0,1] for ...
0
votes
0answers
13 views
How to use custom classifiers in ada?
I am using adaboost from package ada to fit a model. In my understanding of adaboost, it combines multiple classifiers to get a better result. But in the function ...
2
votes
1answer
59 views
probability distribution of output value with regression tree methods
If I have a regression problem where I try to estimate the value of $y$ as function of $x_1 \dots x_d$:
$$
y = f(x_1,\dots,x_d)
$$
using a Boosted Regression Tree or a Random Forest Regression, is it ...
2
votes
2answers
151 views
Random forests vs boosting
I thought it would be interesting to talk about two of the best ensemble methods off-the-shelf: Random Forests and Boosting.
When would you apply one method rather than the other one?
1
vote
1answer
151 views
Using Adaboost for feature selection?
Is it okay to use Adaboost to do feature selection (selecting a subset of dimensions $S$ from a high-dimensional feature vector $V$)?
I divided the samples into four non-overlapping sets: $A$ ...
0
votes
0answers
30 views
Is there a theoretical basis for the shrinkage used in Boosted Regression Trees?
In Gradient Boosted Regression Trees, a shrinkage $\nu$ is often applied as:
$$ f_t(x) \leftarrow f_{t-1}(x) + \nu h(x)$$
where $h$ is the regression tree learned by fitting the tree to the gradient. ...
1
vote
1answer
107 views
How to combine a SVM classifier and a Naive Bayes classifier
I have two different set of features for which I have a SVM classifier and a Naive Bayes classifier, respectively. If I wanted to combine these two classifiers to get a better prediction, what option ...
2
votes
1answer
136 views
Interaction depth parameter in GBM
In the GBM package one is supposed to be able to provide interaction.depth>2, which means higher-order interactions between features.
However, the resulting trees (as seen by pretty.gbm.tree) never ...
4
votes
0answers
80 views
Regarding the sampling procedure in Adaboost algorithm
The AdaBoost algorithm states that it is to train a classifier based on the training data according to a weight vector.
Assume the size of training data is N, the weight vector is of dimension N as ...
1
vote
0answers
31 views
Tutorials / examples for multiclass boosting
I want to learn the multiclass boosting technique. I have a basic understanding of binary boosting and also have seen some working examples on this. I have also read about the basics of multiclass ...
3
votes
1answer
172 views
How to use CART for AdaBoost?
I am trying to use CARTs (Classification and Regression Trees) for AdaBoost as weak learner. My question concerns the update of the weights after fitting the best weak learner.
A single CART node ...
1
vote
0answers
113 views
Adaboost feature weight calculation
I thought I understood Adaboost, until code analysis made me realize that sample_weight is not an array of the feature weights... and after further investigation I am left confused as to how ...
2
votes
1answer
715 views
What is the difference between AdaBoost.M2, AdaBoost.M1, Gentle AdaBoost, RealAdaboost and the Original Adaboost?
I know some questions have been made for example between gentle and ada, but I was wondering about gentle and the others.
For instance, I saw two different works talking about AdaBoost.M1 and ...
1
vote
0answers
62 views
How to modify RankBoost to maximize area under recall-precision curve instead of AUC?
Using the WeakLearn algorithm from the original RankBoost paper, how do you set the optimal threshold to maximize AU-RPC (instead of AUC)? And, once that threshold is set, how do you calculate the ...
0
votes
1answer
96 views
How do I do multiscale HMM classification?
I'm using hidden Markov models to classify some accelerometer data. I take the Fourier transform of the raw data at a given window length, and then train an HMM for each class, and every test instance ...
4
votes
1answer
260 views
Common weak learners for Adaboost
I'm looking for a set of weak classifiers that work with Adaboost to test on popular datasets.
Most of the examples on the web use some kind of random weak learners which work on their own randomly ...
2
votes
1answer
133 views
Concept of iterations in Adaboost
I can't seem to get my head around "iterations" in Adaboost.
Are they analogous to weak classifiers that are used for Boosting?
I've seen many examples of Adaboost where a programmers use a
Single ...
0
votes
1answer
89 views
Accuracy of classifiers with Adaboost
Does Adaboost ensure that resultant accuracy is more than or at least equal to current accuracies?
What happens if Classifier A performs badly and the weights are accordingly updated and the next ...
2
votes
1answer
210 views
How to determine whether a classifier like adaboost is weak?
I run the cross-validation experiment for a given data set, and tried two different approaches: one is based on SVM, another is based on SVM plus Adaboost. But the confusion matrix for two experiments ...
3
votes
0answers
267 views
Classification with GBM in R and imbalanced class sizes
I'm dealing with a supervised binary classification issue. I'd like to use the GBM package to classify individuals as uninfected/infected. I have 15 times more uninfected than infected individuals.
I ...
4
votes
1answer
173 views
Event classification in Higgs' Boson discovery
I am watching the seminar (around min 20) from CERN about Higgs' Boson discovery.
They quickly go on event classification talking about boosting decision trees. I am not really aware on what they ...
2
votes
0answers
473 views
Using AdaBoost on multi-class in R on unbalanced data
I have a data set which is highly imbalanced and I have used the SMOTE algorithm (using the R package DMwR) to balance the binary class in the data set. I have been using the R Ada package to then ...
0
votes
1answer
237 views
Using Adaboost with SVM for classification
I know that Adaboost tries to generate a strong classifier using a linear combination of a set of weak classifiers.
However, I've read some papers suggesting Adaboost and SVMs work in harmony (even ...
2
votes
1answer
259 views
Regarding boosting, bagging and bootstrapping [closed]
How to understand the relationships, comparative advantages, and comparative disadvantages of boosting, bootstrapping and bagging in terms of their respective applications in data mining.
2
votes
1answer
207 views
Selecting features using Adaboost
How does Adaboost select best features from the sample data (or a unit feature vector)?
It would be nice if someone can explain if the above statement is true or not.
I've seen the term features and ...
5
votes
2answers
181 views
Base classifiers for boosting
Boosting algorithms, such as AdaBoost, combine multiple 'weak' classifiers to form a single stronger classifier. Although in theory boosting should be possible with any base classifier, in practice it ...
2
votes
1answer
131 views
What are the strongest boosting alternatives to Adaboost?
Whenever boosting is brought up, Adaboost is the first algorithm to be listed. What are the most popular boosting algorithms that aren't Adaboost?
4
votes
1answer
70 views
Continuous Multistate Ada-boost method?
I recently read this simple introduction to Adaboost as a review after learning about it a few years ago. This is in preparation for actually using it to solve a problem I am working on which is ...
9
votes
2answers
356 views
In boosting, why are the learners “weak”?
See also a similar question on stats.SE.
In boosting algorithms such as AdaBoost and LPBoost it is known that the "weak" learners to be combined only have to perform better than chance to be useful, ...
0
votes
1answer
239 views
Is additive logistic regression equivalent to boosted decision stumps?
Are additive logistic regression and boosted decision stumps (where a decision stump is a one-node decision tree) equivalent in some sense? I thought not, but if I google for "LogitBoost" algorithms, ...
2
votes
1answer
516 views
Does ensembling (boosting) cause overfitting?
I'm using SPSS Statistics Base 20. Using Analyze ==> Regression ==> Automatic Linear Modeling I've input about 50 variables.
When using no boosting, the reported accuracy of the model is 21%. The ...
5
votes
1answer
635 views
Is AdaBoost less or more prone to overfitting?
I have read various (seemingly) contradicting statements whether or not AdaBoost (or other boosting techniques) are less or more prone to overfitting compared to other learning methods.
Are there ...
1
vote
0answers
201 views
Obtaining resampling based estimates of prediction error in boosted regression tree model
I try to use the gbm.fit() function for a boosted regression tree model implemented in the R package gbm. To investigate e.g., the bootstrapped prediction error and ...
4
votes
0answers
136 views
Tree size in gradient tree boosting
Gradient tree boosting as proposed by Friedman uses decision trees with J terminal nodes (=leaves) as base learners. There are a number of ways to grow a tree with ...
5
votes
1answer
294 views
Understanding similarity sensitive hashing algorithm in AdaBoost
I'm a CS major and don't quite understand the mathematics behind a optimization problem coming from a machine learning algorithm. The algorithm is in Section 5 of the paper ...
3
votes
2answers
231 views
What to do when weak classifiers are almost identical in AdaBoost?
I have written some code which uses AdaBoost to generate a set of weak classifiers. I'm finding, however, that when I use the resulting strong classifier to classify examples, it seems that almost ...
7
votes
2answers
1k views
What does interaction depth mean in GBM?
I had a question on the interaction depth parameter in gbm in R. This may be a noob question, for which I apologize, but how does the parameter, which I believe denotes the number of terminal nodes in ...
2
votes
1answer
114 views
Different optimal number of boosting iterations obtained from OOB and on test
If I'm using a machine learning model (e.g. boosted regression trees like gbm in R) on a dataset, what does it mean if there's a significant difference between the OOB estimated optimal # of ...
11
votes
2answers
457 views
On the “strength” of weak learners
I have several closely-related questions regarding weak learners in ensemble learning (e.g. boosting).
This may sound dumb, but what are the benefits of using weak as opposed to strong learners? ...
3
votes
1answer
131 views
AdaBoost on a continuum of base classifiers
A tutorial on AdaBoost suggests that AdaBoost can be applied to a continuum of classifiers (at the bottom of the first page). Does it mean to simply discretize the classifiers, for example, which are ...
2
votes
1answer
184 views
The upper bound of the training error of AdaBoost
I am reading an overview of AdaBoost written by Schapire, which calculates the upper bound of the training error in Eq. (5), section 3. In fact, it states that
...
2
votes
1answer
291 views
How to choose the 1st threshold/classifier/ weak learner in Adaboost?
I am having some difficulty understanding Adaboost.
How should the 1st threshold/classifier/weak learner be chosen?
It seems that there are two conditions which must be satisfied
Choose the ...
1
vote
1answer
132 views
Boosting for regression systems
I am quite a newbie in this area:
What are the boosting methods for regression systems? I know about Gradient boosting; are there any other approaches?
Are there textbooks or tutorials devoted to ...
5
votes
1answer
644 views
When would one want to use AdaBoost?
As I've heard of the AdaBoost classifier repeatedly mentioned at work, I wanted to get a better feel for how it works and when one might want to use it. I've gone ahead and read a number of papers and ...
2
votes
1answer
734 views
Adjusting sample weights in AdaBoost
I am trying to read up about AdaBoost from Tibshirani (page 337 onwards), and would appreciate some help in understanding it better.
The book says that "For each successive iteration m = 2, 3, . . ...
3
votes
0answers
221 views
Normalized sigmoid loss function for boosting?
It seems non-convexity of loss function is not such a problem for boosting with a normalized sigmoid loss function. Do you know any further work showing better results with this kind of boosting than ...
