Boosting is a process of finding & combining weakly predictive models into a strongly predictive model.

learn more… | top users | synonyms

0
votes
0answers
20 views

java.lang.OutOfMemoryError using bartMachine package in R [migrated]

I ran a BART model with 11000 samples and 20 features(half of them are categorical variable). My mac has 8G ram. At first, I set memory to 5000 MB via function set_bart_machine_memory(5000). Then I ...
1
vote
0answers
37 views

Has anyone publicly shared an implementation of RUSBoost in R?

There's no package available on CRAN, so I was hoping someone in the community had written their own function/package. I see it's been done in MATLAB, so I may just have to start with that and write ...
1
vote
0answers
25 views

Classifiers that support weighting of the instances

I'm thinking of implementing my own boosting algorithm, so I'm looking for any multiclass classfication algorithms that would support weighting of the examples, i.e. you could specify what examples ...
0
votes
0answers
26 views

Adaboost for neural networks. Is it still worth it?

I have a question about Adaboost and neural networks. Given the recent development in neural networks (dropout, maxout, or rectified linear units) is there a significant benefit of performing Adaboost ...
1
vote
0answers
46 views

Adaptive Boosting vs. SVM

I am working on a binary classification case and comparing the performance of different classifiers.Testing the performance of adaboost algorithm (with decision ...
2
votes
1answer
149 views

Why not always use ensemble learning?

It seems to me that ensemble learning WILL always give better predictive performance than with just a single learning hypothesis. So, why don't we use them all the time? My guess is because of ...
2
votes
1answer
97 views

Why should all Cross-Validation results be higher than the result on the test dataset?

Sorry, I'm not an expert and my question could be fundamentally wrong. I've read this interesting question because I also was wondering whether to train the model again after cross-validation. Now, ...
0
votes
0answers
14 views

How to extract the importance of predictors in Adaptive boosting?

I have trained a boosting classifier using ada package in R. Now I want to see the importance of my predictors in constructing ...
0
votes
0answers
16 views

boosting with ada package. How do I take the most probable answer from predict?

I trained classifier using ada. Now I executed: predict(adaDol,newdata=cords()) and received response: ...
0
votes
0answers
9 views

Is it possible that boosting doesn't increase the predictive power of tree?

I have a data set of 282 observations, and my response variable is a binary variable where 0=normal and 1 =disease. I constructed a classification tree with rpart ...
0
votes
0answers
25 views

Adaboost and factor variables

My dataset contains both numerical and categorical features like education level, region etc (i use factor variable type for them). I think these variables are important for predicting the outcome of ...
0
votes
0answers
51 views

BRT analysis using count data

I have some problems with my BRT analysis. Introduction to the data: The dependent variable is count data of a specific palm species in SA, and the predictors consists of nine various kinds of ...
3
votes
2answers
147 views

How to find optimal values for the tuning parameters in boosting trees ?

I realise that there are 3 tuning parameters in the boosting trees model, i.e. the number of trees (number of iterations) shrinkage parameter number of splits (size of each constituent trees) My ...
1
vote
0answers
60 views

How is L2 Boosting Different from a Big Regression Tree?

I'm learning about boosting. I think I understand how adaptive boosting works for classification. I'm trying to get some intuition for regression boosting. At each iteration, adaptive boosting ...
0
votes
0answers
21 views

how to generate samples for model selection testing on adaboost

I want to assess some statistical methods for model selection on binary classification using adaBoost. For this, I have to generate artificial samples (input data) and create an oracle that has the ...
0
votes
1answer
58 views

Values of the weights in Adaboost

I have implemented a simple Adaboost algorithm, using several weak classifiers, and when checking the values computed by it there are alphas with a negative value. Is that possible, or is there a bug ...
0
votes
1answer
59 views

dealing with exponentials in python - infinities and overflows [duplicate]

In a machine learning algorithm that I'm using, I need to get the exponential values of something in one of the steps. This is the step that I'm dealing with right now: I've already got all the ...
0
votes
0answers
41 views

Understand the parameters for multiple regression - question based on the notations in a given algorithm

I've set up a regression model but am not sure if I'm doing it right. I'm using multiple regression to help do multi-class classification. So far I feel like I've understood the theory, but I'm ...
0
votes
1answer
312 views

Boosted decision trees using Matlab

I would like to experiment with classification problems using boosted decision trees using Matlab. In the paper An Empirical Comparison of Supervised Learning Algorithms this technique ranked #1 with ...
4
votes
1answer
161 views

What does it mean to “fit a regression function” and then use it to update other functions?

Referring to the algorithm on page 11 in this paper on boosting algorithms, I really don't understand step 2, (ii) and (iii). What does this mean: (ii) Fit the regression function $g_j^h (x)$ by ...
0
votes
0answers
13 views

Boosting in Weka with single PC

Given that I want to run Weka in a single PC, is it a good a idea to use boosting with a three layer feedforward neural network as a base learner? Why or why not? I think boosting requires a lot of ...
2
votes
0answers
68 views

BRT predictions on zero-inflated gaussian fish abundances include negative results

hopefully someone can point me in the right direction here. I'm using boosted regression trees (BRT) to assess the relative importance of a number of environmental factors (sea bottom temperature, ...
1
vote
1answer
845 views

R: What do I see in partial dependence plots of gbm and RandomForest?

Actually, I thought I had understood what one can show a with partial dependence plot, but using a very simple hypothetical example, I got rather puzzled. In the following chunk of code I generate ...
1
vote
0answers
24 views

weighted conditional expectation in adaboost

I'm looking at this paper, http://www.stanford.edu/~hastie/Papers/AdditiveLogisticRegression/alr.pdf around pages 10-11 (marked 346,347 in the pdf). This notation is introduced $$ E_w[g(x,y)|x] := ...
0
votes
0answers
53 views

Adaboost: weak learn on different dataset for each iterations

I have seen a few occasion were a weak learner was used iteratively with different dataset. For example, let's say you want to learn to classify {1, -1} according to 3 properties and naive Bayes weak ...
0
votes
1answer
31 views

Applying ensemble learning to quantile regression?

Is it desirable / possible to apply ensemble learning methods (boosting, bagging, etc) to the quantile regression problem?
0
votes
0answers
17 views

what does Task in multitask learning mean?

I'm new in multitask learning. I didn't exactly understand what is Tasks in multitask learning ? every task are single training data or what ?
0
votes
1answer
103 views

Shrinkage parameter in Adaboost?

I'm unclear how the shrinkage parameter works in Adaboost. I understand the concept of shrinkage in the theoretical sense related to ordinary least squares, but I'm not sure how to interpret this ...
0
votes
0answers
64 views

Default prediction threshold for two-class gradient boosting classification

At the bottom of page 9 in http://statweb.stanford.edu/~jhf/ftp/trebst.pdf, the author relates the final function $F_{m}$ to $P(y=1|x)$ via the logistic function, and suggests classifying a data point ...
1
vote
2answers
116 views

How Gradient boosting can be more interpretable than CART?

I found this document which compare some learning methods and I don't understand this table : Gradient boosting has a better intepratability score than CART. How is it possible ? I thought gradient ...
1
vote
0answers
76 views

Gradient boosting algorithm (steps) question

So, far I have read following regarding boosting: Boosting is an ensemble technique. Train learner sequentially, where early learners fit simple models to the data. Analyze data for errors, that ...
0
votes
1answer
98 views

C5.0, boosting, and mislabeled data

I'm trying to model a binary classification problem. 5 continuous features, slightly imbalanced dataset (about 60 in one class, a little more than 200 in another). So far I've tried kNN, LDA, and ...
7
votes
4answers
942 views

Random Forest, is it a boosting algorithm?

Short definition of boosting: Can a set of weak learners create a single strong learner? A weak learner is defined to be a classifier which is only slightly correlated with the true ...
0
votes
1answer
173 views

Gradient boosting in R uses only a single variable

I am trying to build a boosting model using the package gbm in R. I have the following code: ...
3
votes
1answer
78 views

Multiple definitions of AdaBoost

The description of AdaBoost in Kevin Murphy's Machine Learning book (shown in a snapshot below) differs from the one in Wikipedia. I am trying to relate both definitions. Step by step, my questions ...
0
votes
1answer
297 views

boosting with linear svm

I am working on boosting classifier. I am planning to use linear svm as the weak classifier. I am using liblinear for it. My question is how can I weight each instance of liblinear based on the ...
0
votes
0answers
42 views

Minimizing Specificity in Adaboost (and other) Classifiers

This is a relatively simple question, but is not addressed in the documentation for the ada package in R. How do I go about instructing ...
0
votes
0answers
404 views

C5.0 Node Decision Tree with Boosting Algorithm (SPSS Modeler)

We are currently looking to mimic the SPSS Modeler's C5.0 Node decision tree model with boosting, which means we'd like to rewrite the whole thing in plain Java. I ...
0
votes
1answer
63 views

componentwise boosting based on fisher scoring

componentwise boosting dates back at least to Bühlman and Yu (2003), where in each boosting iteration a set of base-learners (e.g. simple linear models) depending on a subset of the covariates are ...
1
vote
0answers
51 views

IRLS working weights proportional odds

According to Reduced-rank vector generalized linear models the parameter estimates are obtained by Fisher-scoring algorithm. In the VGAM package the IRLS algorithm ...
2
votes
1answer
136 views

Boosting algorithms and overfitting

In the book of Witten and Frank on Data Mining algorithms, I read: "If boosting succeeds in reducing the error on fresh test data, it often does so in a spectacular way. One very surprising finding ...
3
votes
1answer
123 views

Why is boosting effective?

Boosting takes a bunch of weak learners and creates a strong learner. But why is it so difficult to create a strong learner right from the beginning without using boosting techniques? And therefore ...
2
votes
1answer
124 views

Is there a well-defined class of ensemble methods?

Ensemble methodology's main aim is to somehow aggregate or summarize estimates from multiple models. In some cases this is aggregating different bootstrap estimates or Monte Carlo estimates, but ...
1
vote
0answers
106 views

GBDT and model building: How am I overfitting?

Here's my situation: Binary classification and I've got a training set of roughly 250k samples and 10 features, and a validation set of roughly 100k with the same number of features. I'm fitting GBDT ...
1
vote
0answers
41 views

Adaboosted trees and q-learning

Is there a solid mathematical connection between ADABOOST and Q-learning that can inform "higher performance" machine learning algorithms? Q-learning references: ...
1
vote
1answer
147 views

The weight updating in adaboost

1.AdaBoost updates the weight of the sample By the current weak classifier in training each stage. Why doesn't it use the all of the previous weak classifiers to update the weight. (I had tested it ...
2
votes
1answer
56 views

How can I do logistic correction for boosting

Can anyone tell me if logistic correction is the best method to correct the probability of gradient boosting machine? If so, how can I do it?
1
vote
0answers
90 views

Which Regression methods are suitable for binary valued features and continuous output?

I want to build a machine learning model to regression on continuous output given binary valued features(0,1). the dimension of my problem is around 200. which of the flowing methods seems suitable ...
1
vote
4answers
183 views

Measuring representativeness of a sample using covariates

I was provided with quite a small sample of labeled (variable of interest) observations to train a model to predict unlabeled observations. All the observations are associated with many covariates. ...
2
votes
1answer
116 views

probability distribution of output value with regression tree methods

If I have a regression problem where I try to estimate the value of $y$ as function of $x_1 \dots x_d$: $$ y = f(x_1,\dots,x_d) $$ using a Boosted Regression Tree or a Random Forest Regression, is it ...