Skip to main content

Questions tagged [adaboost]

A popular boosting algorithm (short for "adaptive boosting"). Boosting combines weakly predictive models into a strongly predictive model.

Filter by
Sorted by
Tagged with
1 vote
1 answer
30 views

Can AdaBoost discriminate the data?

Assume that you have an image $X$ and you want to search a specific small object inside the image $X$. It's only one object. So you training an AdaBoost model with only one valid train sample and the ...
euraad's user avatar
  • 425
1 vote
1 answer
30 views

AdaBoost in Matlab - Only get 88% accuracy with Fisher's Iris data set while 100% in one layer Neural Network

I have made AdaBoost in Matlab. I get 88% in accuracy when I use Fisher's Iris flower set data. Here is the working example: ...
euraad's user avatar
  • 425
0 votes
0 answers
13 views

Linear SVM vs Decision Stumps for AdaBoost

I have heard that AdaBoost can use a linear SVM as a weak classificer. I wonder why Decision Stumps is often used with AdaBoost? Booth are binary classifiers. In my opinion, linear SVM seems to be a ...
euraad's user avatar
  • 425
0 votes
0 answers
10 views

What's the purpose by returning back the bad outliers from AdaBoost?

Assume that you having a matrix $X$ that holds both numerical and binary data. You plug in the data $X$ into AdaBoost and AdaBoost update the $X$ by only focusing on the rows that could not make a ...
euraad's user avatar
  • 425
2 votes
0 answers
47 views

Confusion about the code for choosing "stumps" in Adaboost algorithm

(I actually asked the following question on Stack Overflow recently: https://stackoverflow.com/questions/76842431/confusion-about-the-code-for-choosing-stumps-in-adaboost-algorithm but then I ...
Richard's user avatar
  • 121
2 votes
0 answers
134 views

AdaBoost vs GBM vs XGB [closed]

Can someone please summarise the difference between these 3 boosting algorithms: AdaBoost vs GBM vs XGB?
Amina Umar's user avatar
3 votes
1 answer
151 views

Does boosting being component-wise imply variable selection?

I have come across the term "component-wise" in the literature, and I am curious if this means that a model does perform variable selection. And not having this, mean it doesn't perform ...
paulmuaddib's user avatar
1 vote
0 answers
134 views

Gradient Boosting interpretation of residuals

So I came across this sentence (from an e-learning website): By training on the residuals of the model, this is an alternative means to give more importance to misclassified observations. Intuitively,...
Caterina's user avatar
  • 229
1 vote
0 answers
43 views

Adaboost updating the weight. How to use Induction to proof two equation are the same?

I have these equation for updating the weight the same. $$w^{(b+1)}_{i} = \frac{w_i^{(b)} \exp(-\alpha_by_ih_b(x_i))}{\sum_{j=1}^{n}w_j^{(b)} \exp(-\alpha_by_jh_b(x_j))}$$ and $$w^{(b+1)}_{i}=\frac{\...
Kimbly  North's user avatar
0 votes
0 answers
158 views

Why `adaboost` using `gbm` package will not produce exact binary prediction values such as `{1,0}`?

My input data contains a response variable is_disease which is obviously a Factor(Yes,...
nobodyishere's user avatar
1 vote
0 answers
82 views

Model which achieves good results on the Adult dataset, especially on the minority class

I'm looking for a model which achieves an high prediction and recall on the Adult dataset. I've tried with random forests, but couldn't get a model with good results on the minority class. Here are my ...
Gerardo Zinno's user avatar
1 vote
1 answer
207 views

What’s the purpose of the 0.5 in the alpha term in Adaboost?

In the AdaBoost algorithm, the contribution of each weak learner is multiplied by a weight $\alpha$ given by $$ \alpha_t = \frac{1}{2}ln\Big(\frac{1-\epsilon_t}{\epsilon_t}\Big) $$ In the Elements of ...
user1809989's user avatar
1 vote
0 answers
54 views

Is there a stochastic AdaBoost?

In sklearn, the AdaBoostClassifier and AdaBoostRegressor classes do not have the subsample and max_features parameters, which are responsible for the stochastic approach to building a boosting model. ...
Владислав Гаджиханов's user avatar
1 vote
1 answer
126 views

Boosting usa bootstraping?

I had a question about boosting. When in the first iteration of the algorithm we pass our data to the first decision tree, this data we pass is a sample generated by bootstraping or is it the original ...
PicaPython's user avatar
4 votes
2 answers
868 views

Can decision stumps have more than 2 leaves?

I understand decision stump: a shallow 1-level decision tree is often used as base-leaner in ensemble methods such as AdaBoost. What is not immediately clear to me ...
arilwan's user avatar
  • 273
1 vote
1 answer
98 views

Divergence of accuracies and overfitting in AdaBoost

I implemented a binary classification setup of AdaBoost, but I train one model for each label in a one-vs-all arrangement, and in the prediction time I choose the class corresponding to the model with ...
abbassix's user avatar
  • 131
1 vote
0 answers
46 views

Oscillation of AdaBoost Training error

Adaboost, using weak learners as Gaussian Naive bayes, has oscillating/unpredictable training error as we increase the number of weak learners. Is there a specific reason for this? Y-axis is the ...
A J's user avatar
  • 29
5 votes
1 answer
2k views

Boosting reduces bias when compared to what algorithm?

I am reading on bagging and boosting, and I understand how they both work (at least I think I do). I would like to talk in the context of decision tree ensembles as I think (not sure if correct) that ...
Slim Shady's user avatar
1 vote
1 answer
71 views

How to adjust sample weights in Adasboost

I am following this video tutorial to understand Adaboost I am confused about the sample weights updating. It first calculates the amount of say of each stump by this formula, where total error is ...
ycenycute's user avatar
  • 373
3 votes
2 answers
266 views

Where is Boosting applied in Gradient boosting techniques?

In boosting, the primary idea is to re-adjust weights of training instances, so that subsequent models learn how to fit difficult-to-classify samples. From Wikipedia Boosting (Machine Learning): ...
The Wanderer's user avatar
1 vote
1 answer
84 views

how train AdaBoost M1 weak estimators?

I'm trying to implement AdaBoost.M1 as explained in Boosting: Foundations and Algorithms by Robert E. Schapire and Y. Freund. The problem is that I don't understand at each iteration t the estimator ...
abbassix's user avatar
  • 131
0 votes
0 answers
61 views

Equation 10.2 from The Elements of Statistical Learning. Median of a chi-squared distribution

I'm reading about AdaBoost in the The Elements of Statistical Learning and I don't understand the equation 10.2. Below is an excerpt from the book. The power of AdaBoost to dramatically increase the ...
Jedrek369's user avatar
0 votes
1 answer
456 views

Prediction of Adaboost Classifier between 0 and 1

I have to use an Adaboost classifier to predict if the data is a signal (1) or a background (0) event. But since the output is expected to be 1 or 0 the ...
Wolfmercury's user avatar
2 votes
0 answers
215 views

Generalisation error bound proof for the Adaboost algorithm, corollary 4.4 in Freund and Schapire (2014)

I am having difficulty with a detail in the proof of generalisation error bounds for the AdaBoost algorithm, and would appreciate some assistance. This is the proof of corollary 4.4 on page 54 of ...
microhaus's user avatar
  • 2,550
5 votes
1 answer
633 views

Why does classifier (XGBoost) "after PCA" runtime increase compared to "before PCA"

The short version: I am trying to compare different classifiers for a certain dataset from kaggle, and am trying to also compare these classifiers between before using PCA (form sklearn) to after ...
appeldaniel's user avatar
0 votes
0 answers
358 views

How does resampling in AdaBoost (exactly) work?

Overall, I like to think that I understand how AdaBoost works, i.e., fitting a weak learner, calculating the error, calculating the confidence / amount of say of the learner, updating the sample ...
Christian's user avatar
  • 197
1 vote
1 answer
3k views

What's the purpose of learning rate in sklearn AdaBoost implementation

We know that sklearn's implemenation of AdaBoost algorithm uses DecisionTreeClassifier as the base learner. Conceptually, ...
arilwan's user avatar
  • 273
1 vote
0 answers
126 views

Can I use AdaBoost Regressor with Gaussian Process Regressor as base estimator, as my Surrogate Function?

I am using a text-extraction model whose hyperparamters I want to optimise. The model takes time(on average 1 hr) for training on the dataset. The algorithm I am using for hyperparameter tuning, is ...
supratim saha's user avatar
1 vote
0 answers
152 views

Interpretation of a tightly paired learning curve with increasing loss

I am assessing models for a binary classification task and have created a model with a very strange learning curve. This is the learning curve of an sklearn AdaBoostClassifier fitted with default ...
callum's user avatar
  • 11
0 votes
1 answer
315 views

Why is AdaBoost with short decision trees a form of feature selection?

Why is AdaBoost with short decision trees a form of feature selection? What is so special with short decision trees?
Agus Bendoro's user avatar
6 votes
2 answers
4k views

AdaBoost - why decision stumps instead of trees?

Since the original AdaBoost article it has been found out that boosting reduces both variance and bias in the classifier (in contrast to bagging, which only reduces variance). Original AdaBoost (and ...
qalis's user avatar
  • 238
1 vote
1 answer
625 views

Why does AdaBoost algorithm use weighted data points?

I am learning about AdaBoost algorithm. At each iteration, adaBoost set higher weight to mistake datapoints, and lower weight to correct classified data points. I do not understand why the algorithm ...
i_love_somebody's user avatar
1 vote
0 answers
28 views

what will be modified algorithm if this loss function will be modified? [closed]

I have a loss function of AdaBoost and $l(h(x),y)=e^{(-y*h(x))}$ and what will be modified algorithm if we modify loss function to $l(h(x),y)=ln(1+e^{(-y*h(x)})$. I was not able to find a way to ...
jack vorobey's user avatar
2 votes
2 answers
1k views

Does Adaboost ensemble use bootstrapping?

I am reading about boosting methods in the book Elements of statistical learning. In page 339 they describe the Ada boost algorithm as I understand the general idea behind it: Give more weight to ...
Alberto Perez Martinez's user avatar
0 votes
1 answer
302 views

How to generate a confidence interval for an adaboost prediction?

I have created a simple AdaBoostRegressor (sklearn) model which is trained on a feature set $X$ (house features) to predict a variable $y$ (house prices). The model can be used to create a prediction ...
James T's user avatar
  • 23
1 vote
0 answers
218 views

What is a reasonable number of splits (maximum) for a ensemble classifier?

I am trying to use an ensemble classifier (honing in on Matlab fitcensemble). I've also explored using a single decision tree as well as tree bagging (Matlab fitctree, TreeBagger) Simple binary (A/B) ...
bliswell's user avatar
3 votes
1 answer
214 views

in which part is used the splitting criteria in AdaBoost?

I have been reading the original article about AdaBoost and by comparing with other reading material it has come some doubts about this model. Please feel free to correct me if in any part I am wrong. ...
Layla's user avatar
  • 621
0 votes
0 answers
27 views

Getting bad results when testing my prediction model on new patients

I analyze medical diagnostic labeled data: the independent variables are medical parameters (assume that one of the variables is blood pressure) and the dependent variable is 1 or 0. The data contains ...
Amit S's user avatar
  • 57
0 votes
0 answers
284 views

What is this symbol in Adaboost algorithm (SAMME)?

According to the original paper of SAMME algorithm (Adaboost for multiclass problems), it is described as follows: The general idea is very straightfoward, except for this symbol: The author didn't ...
joann2555's user avatar
  • 143
2 votes
2 answers
1k views

Improve Adaboost that using weighted logistic regression instead of decision trees

I implemented Adaboost that using weighted logistic regression instead of decision trees and I managed to get to 0.5% error, I'm trying to improve it for days with no success and I know it possible to ...
Oran Sherf's user avatar
6 votes
0 answers
117 views

Weighted Conditional Expectation definition in AdaBoost

I am looking at "Additive logistic regression a statistical view of boosting" paper (https://web.stanford.edu/~hastie/Papers/AdditiveLogisticRegression/alr.pdf) In page 346, the authors ...
Aggamarcel's user avatar
1 vote
2 answers
811 views

Adaboost -- how does reweighting affect the learning process for the subsequent learner?

In Adaboost, when you reweight the samples, how does the training process for the next classifier in the boosting algorithm take in to account the weights? Is it reflected in the loss function of the ...
student010101's user avatar
3 votes
2 answers
1k views

Random forest after cross validation

i have been wondering for some time now how random forests (or AdaBoost, doesn't matter) are built when using cross-validation. Let's see we're using 5-fold cross validation to train random forests on ...
Steven Niggebrugge's user avatar
1 vote
1 answer
731 views

Weights in Adaboost

The Adaboost algorithm is shown below. I have a couple of questions. First, I am a little confused on how the weights for each training sample is applied when using Adaboost. From looking at the ...
David's user avatar
  • 715
1 vote
1 answer
554 views

AdaBoost assumption of weak classifier

I've read and think I got a good grasp of the math behind AdaBoost, but I wasn't able to understand why AdaBoost requires a weak base-classifier? Specifically, I'm dealing with AdaBoost using ...
Ruslan's user avatar
  • 113
1 vote
0 answers
34 views

Creating Predictions From Base Regressor Models in a Boosted Machine

So I'm writing my own implementation of AdaBoost using chapter 10 of Elements of Statistical Learning. The pseudocode given in the book lists how to make the algorithm for a classification problem, ...
Jonathan Bechtel's user avatar
1 vote
0 answers
163 views

AdaBoost Weight Updating for Misclassified Points

http://www.inf.fu-berlin.de/inst/ag-ki/adaboost4.pdf I was reading this paper and came across the question showed in the second image. However, in the first image provided in a book on Machine ...
pineapple69's user avatar
0 votes
0 answers
576 views

Why does AdaBoost use decision stumps instead of 0-depth trees?

Why is it that AdaBoost uses decision stumps for the weak learners? It seems simpler to me to just use the weighted majority of the data points for the classification. Why shouldn't we do this?
MLNewbie's user avatar
0 votes
0 answers
363 views

How to update the weight when Adaboost is used for regression

When Adaboost is used for regression, in the $m$ step, we suppose $G_m(x)$ is $m$-classifier with sample $\{(x_i,y_i)\}$; the error of i-th sample is $e^i_m;$ the weight of $e^i_m$ is $w^i_m;$ the ...
user6703592's user avatar
  • 1,345
0 votes
0 answers
298 views

Termination Condition for AdaBoost.R2

I can't quite wrap my head around the termination condition of AdaBoost.R2 as defined by Drucker in this paper. On page 2 of the paper he states to "repeat the following while the average loss* $\bar{...
Sigmaeon's user avatar