Tagged Questions
1
vote
1answer
35 views
The weight updating in adaboost
1.AdaBoost updates the weight of the sample By the current weak classifier in training each stage. Why doesn't it use the all of the previous weak classifiers to update the weight. (I had tested it ...
1
vote
0answers
39 views
Which Regression methods are suitable for binary valued features and continuous output?
I want to build a machine learning model to regression on continuous output given binary valued features(0,1). the dimension of my problem is around 200. which of the flowing methods seems suitable ...
1
vote
4answers
127 views
Measuring representativeness of a sample using covariates
I was provided with quite a small sample of labeled (variable of interest) observations to train a model to predict unlabeled observations. All the observations are associated with many covariates. ...
0
votes
0answers
30 views
Is there a theoretical basis for the shrinkage used in Boosted Regression Trees?
In Gradient Boosted Regression Trees, a shrinkage $\nu$ is often applied as:
$$ f_t(x) \leftarrow f_{t-1}(x) + \nu h(x)$$
where $h$ is the regression tree learned by fitting the tree to the gradient. ...
4
votes
0answers
80 views
Regarding the sampling procedure in Adaboost algorithm
The AdaBoost algorithm states that it is to train a classifier based on the training data according to a weight vector.
Assume the size of training data is N, the weight vector is of dimension N as ...
1
vote
0answers
62 views
How to modify RankBoost to maximize area under recall-precision curve instead of AUC?
Using the WeakLearn algorithm from the original RankBoost paper, how do you set the optimal threshold to maximize AU-RPC (instead of AUC)? And, once that threshold is set, how do you calculate the ...
4
votes
1answer
261 views
Common weak learners for Adaboost
I'm looking for a set of weak classifiers that work with Adaboost to test on popular datasets.
Most of the examples on the web use some kind of random weak learners which work on their own randomly ...
2
votes
1answer
210 views
How to determine whether a classifier like adaboost is weak?
I run the cross-validation experiment for a given data set, and tried two different approaches: one is based on SVM, another is based on SVM plus Adaboost. But the confusion matrix for two experiments ...
3
votes
0answers
268 views
Classification with GBM in R and imbalanced class sizes
I'm dealing with a supervised binary classification issue. I'd like to use the GBM package to classify individuals as uninfected/infected. I have 15 times more uninfected than infected individuals.
I ...
2
votes
1answer
259 views
Regarding boosting, bagging and bootstrapping [closed]
How to understand the relationships, comparative advantages, and comparative disadvantages of boosting, bootstrapping and bagging in terms of their respective applications in data mining.
2
votes
1answer
207 views
Selecting features using Adaboost
How does Adaboost select best features from the sample data (or a unit feature vector)?
It would be nice if someone can explain if the above statement is true or not.
I've seen the term features and ...
5
votes
2answers
181 views
Base classifiers for boosting
Boosting algorithms, such as AdaBoost, combine multiple 'weak' classifiers to form a single stronger classifier. Although in theory boosting should be possible with any base classifier, in practice it ...
4
votes
1answer
70 views
Continuous Multistate Ada-boost method?
I recently read this simple introduction to Adaboost as a review after learning about it a few years ago. This is in preparation for actually using it to solve a problem I am working on which is ...
9
votes
2answers
356 views
In boosting, why are the learners “weak”?
See also a similar question on stats.SE.
In boosting algorithms such as AdaBoost and LPBoost it is known that the "weak" learners to be combined only have to perform better than chance to be useful, ...
5
votes
1answer
635 views
Is AdaBoost less or more prone to overfitting?
I have read various (seemingly) contradicting statements whether or not AdaBoost (or other boosting techniques) are less or more prone to overfitting compared to other learning methods.
Are there ...
1
vote
0answers
201 views
Obtaining resampling based estimates of prediction error in boosted regression tree model
I try to use the gbm.fit() function for a boosted regression tree model implemented in the R package gbm. To investigate e.g., the bootstrapped prediction error and ...
5
votes
1answer
294 views
Understanding similarity sensitive hashing algorithm in AdaBoost
I'm a CS major and don't quite understand the mathematics behind a optimization problem coming from a machine learning algorithm. The algorithm is in Section 5 of the paper ...
3
votes
2answers
231 views
What to do when weak classifiers are almost identical in AdaBoost?
I have written some code which uses AdaBoost to generate a set of weak classifiers. I'm finding, however, that when I use the resulting strong classifier to classify examples, it seems that almost ...
7
votes
2answers
1k views
What does interaction depth mean in GBM?
I had a question on the interaction depth parameter in gbm in R. This may be a noob question, for which I apologize, but how does the parameter, which I believe denotes the number of terminal nodes in ...
2
votes
1answer
114 views
Different optimal number of boosting iterations obtained from OOB and on test
If I'm using a machine learning model (e.g. boosted regression trees like gbm in R) on a dataset, what does it mean if there's a significant difference between the OOB estimated optimal # of ...
11
votes
2answers
457 views
On the “strength” of weak learners
I have several closely-related questions regarding weak learners in ensemble learning (e.g. boosting).
This may sound dumb, but what are the benefits of using weak as opposed to strong learners? ...
3
votes
1answer
131 views
AdaBoost on a continuum of base classifiers
A tutorial on AdaBoost suggests that AdaBoost can be applied to a continuum of classifiers (at the bottom of the first page). Does it mean to simply discretize the classifiers, for example, which are ...
2
votes
1answer
184 views
The upper bound of the training error of AdaBoost
I am reading an overview of AdaBoost written by Schapire, which calculates the upper bound of the training error in Eq. (5), section 3. In fact, it states that
...
2
votes
1answer
291 views
How to choose the 1st threshold/classifier/ weak learner in Adaboost?
I am having some difficulty understanding Adaboost.
How should the 1st threshold/classifier/weak learner be chosen?
It seems that there are two conditions which must be satisfied
Choose the ...
5
votes
1answer
644 views
When would one want to use AdaBoost?
As I've heard of the AdaBoost classifier repeatedly mentioned at work, I wanted to get a better feel for how it works and when one might want to use it. I've gone ahead and read a number of papers and ...
2
votes
1answer
734 views
Adjusting sample weights in AdaBoost
I am trying to read up about AdaBoost from Tibshirani (page 337 onwards), and would appreciate some help in understanding it better.
The book says that "For each successive iteration m = 2, 3, . . ...
7
votes
1answer
585 views
Calibrating a multi-class boosted classifier
I have read Alexandru Niculescu-Mizil and Rich Caruana's paper "Obtaining Calibrated Probabilities from Boosting" and the discussion in this thread. However, I am still having trouble understanding ...
5
votes
1answer
275 views
When is the shrinkage applied in Friedman's stochastic gradient boosting machine?
In boosting, each additional tree is fitted to the unexplained variation in the response that is currently un-modelled. If we are using squared-error loss, this amounts to fitting on the residuals ...
8
votes
1answer
1k views
How does gentle boosting differ from AdaBoost?
There is a variant of boosting called gentleboost. How does gentle boosting differ from the better-known AdaBoost?
12
votes
3answers
693 views
How boosting works?
What is the easiest way to understand boosting?
Why doesn't it boost very weak classifiers "to infinity" (perfectness?) ?
