The tag has no usage guidance.

learn more… | top users | synonyms

0
votes
1answer
22 views

How are the outcomes that generated from different predictive models combined to get more accurate predictions?

The simple average is commonly used to combine the predictions of different predictive models. Apart form the simple average, what are the other methods that can be used for combining the predictive ...
2
votes
1answer
44 views

Can I use output of classifier A as feature for classifier B?

This is likely to be a confused question, but I'm curious if this is a valid way to combine classifiers. I have a classification data set, i.e. column of labels and N columns of features, and I use a ...
0
votes
1answer
15 views

Is it appropriate to exclude outliers in ensemble learning?

While the term 'appropriate' is usually subjective and largely dependent upon the domain that one is talking about, I am here asking whether the exclusion of outliers goes against the principle of ...
1
vote
0answers
9 views

Understanding Plurality Voting in Ensemble Methods

I'm reading Data mining with decision trees by Rokach, and i've got to a chapter about ensemble methods (using multiple classifiers) and this is where I can't wrap my head around this concept of ...
1
vote
0answers
43 views

Why do boosting overfit on the data with uniform noise?

i read about it but i didn't get the idea, and actually i didn't find many pages that talk about uniform noise with boosting, is it rare to happen or what? another question: i read in some pages that ...
1
vote
0answers
38 views

Isn't stacking models a direct approach to overfitting?

With help by the discussions here I successfully trained various models for classification. As an example say I trained a stochastic gradient boosted model (gbm) and an extreme gradient boosted tree ...
0
votes
0answers
16 views

Combining binary classification algorithms

I have several algorithms which solve a binary classification (with response 0 or 1) problem by assigning to each observation a probability of the target value being equal to 1. All the algorithms try ...
0
votes
1answer
23 views

If you have several models each based on a subset of features, how can you combine the models to make a better prediction?

As a simple example, say you want to predict the house price. You have 5 features. You build 5 models, each trained with one feature. (price, sqft) (price, num_bedrooms) (price, lot_size) (price, ...
0
votes
0answers
9 views

Should we predict residual of ensemble model and add it to final prediction?

We have been doing a time series project on daily wise data. We built 5 different models SVM, XGBoot, ARIMA, KNN, ANN. We then built predictive models for residuals for all of these models and got ...
1
vote
1answer
47 views

Scalable Random Forest For Massive Data

My problem is simple. I want to train a dataset using random forest on a huge dataset (with $n$ rows). Let's assume I can only fit $b < n$ rows in memory at a time. Model Choice I see several ...
1
vote
1answer
13 views

Simple way to combine predictions from multiple classifiers?

I have predictions from 3 binary classifiers (SVM, RF and NN) and would like to combine them in some way. I'm aware of the notion of ensemble learning methods but I was wondering if it would be valid ...
3
votes
1answer
130 views

allocating degree belief to forecast value

We have a number of providers for a forecast of wind power generation per country per date. Values are forecast up to one week ahead. Forecasts may be compared with actual values of reported wind ...
1
vote
1answer
22 views

Trees of ensembles.

I have a large dataset (100k+), and it's growing everyday. I want to train it to predict a value (a regression problem). I've been finding that ensemble trees work the best for now, but in the ...
0
votes
0answers
26 views

How do they do classifier ensemble using stacking

I am very new in data science, now I am trying to learn Ensembling with Azure Machine Learning. This example seems interesting Ensemble Classifier using Stacking But when I checked those modules in ...
0
votes
0answers
13 views

How can one quantify the variable importance dilution effect in random forests (and similar statistical learning methods)?

In Applied Predictive Modelling (Kuhn, Johnson, 2013, p 202), the authors refer to a dilution effect whereby compared to a single tree or a classical regression technique, the difference in importance ...
1
vote
1answer
48 views

Why are gradient boosting regression trees good candidates for ranking problems?

I have been reading up on gradient boosting machines, and in particular GBRT's. I've come across numerous mentions (and finally tracked down some papers) on applying these models to ranking problems - ...
0
votes
0answers
11 views

kNN used as a metafeature for future ensemble

Lately I've saw a lot of classification approaches for large datasets that involved ensemble methods, most of them using kNN. If I don't miss-understand the algorithm, it is meant to use a portion of ...
2
votes
1answer
62 views

Ensemble models perform worse than single one

In my model testing, I tried to use model ensembling (blending in this case) to get better results. However the ensemble cannot beat single RandomForrestClassifier. In first layer, I train meta-...
0
votes
0answers
11 views

Can I avoid overfit in an ensemble model using a bootstrap with small samples as training sets?

Let´s suppose that t I have a dataset with 250 data points and I want to train an ensemble. If I choose to train each classifier of the ensemble with a small bootstrap sample (10) of the original ...
1
vote
1answer
51 views

How to proceed with building an ensemble classifier using Naive Bayes, TAN and Logistic Regression in R

I'm relatively new to machine learning (started about 5 months ago), and I'm looking at potentially implementing an ensemble classifier as part of my research. I have built 3 models that I use to ...
1
vote
2answers
40 views

ensemble model of uni-variate linear regression models

Calculating univariate linear regression and correlation is simple in SQL. It can be done like so: ...
2
votes
0answers
18 views

Multiclass classification one versus one with ensemble

If I use an ensemble, which consists of four classifiers, in order to classify my data into three classes. Further, suppose I use the one versus one strategy. My question is: How to fuse the ...
0
votes
0answers
13 views

Combining classification results

Let's say I have a binary classification problem and several models $M_1, \dots, M_n$ to predict the classification results. In my setting the outputs of these models are quantitative (some of them ...
1
vote
0answers
42 views

ensemble methods:voting with average of probabilities in weka

output attribute is risky patient. Values are yes and no.if yes then patient is risky and if no then patient is not risky. If I am combining 3 classifier for classification model in weka, and if ...
0
votes
1answer
16 views

Weighing classifiers based on the cross validation accuracy

Suppose there exist n individual classifiers which have different parameters and have been trained on the same data. In order to build an ensemble of these classifiers, is it an optimal method to ...
0
votes
0answers
45 views

How to evaluate stacking ensemble model vs. other models with 10-fold cross-validation?

I've been comparing various predictive models for both continuous and binary outcomes for a health care model. So far 10-fold cross-validation has been useful: training models on 9/10 of the analysis ...
1
vote
0answers
25 views

Model selection in ensembles

I'm trying to build an ensemble for a ML problem where fast prediction time is critical. So I'm interested in keeping my set of level-0 models for the ensemble pruned. Which measures can I use to ...
1
vote
0answers
30 views

Boosting in unsupervised learning - methods and use cases

I'm looking for methods and uses cases for applying boosting or other ensemble methods for unsupervised learning Examples of such methods and use cases are: Boosting density estimation Saharon ...
3
votes
0answers
16 views

Theory of correlation and weighing when ensabling models

I'm ensabling models together to improve the overall performance. At the moment, I'm weighing each model by its performance under cross-validation, and this works reasonably well. Clearly the best ...
0
votes
0answers
14 views

Classification: estimate how many people are in a household through account transactions

I want to estimate how many people are in a household by looking at account transactions. It would be also interesting to understand if in the household there are children. I thought that a possible ...
3
votes
1answer
116 views

stacking and blending of regression models

I am self-studying blending and stacking, and am especially interested in this in the context of regression models. I have been reading a number of the stacking, blending and bagging links posted on ...
0
votes
0answers
28 views

Combining strong and weak learners

Our NGO is struggling with the following problem: For an ecological application, we have tried to program a deep neural net to estimate the size and weight of images of birds that we have captured on ...
0
votes
0answers
16 views

How to decide about the number of looks (window size for ensemble averaging) in SAR images?

This question has frustrated me for a while. In order to find an answer I sent an email to prof. Yamaguchi, the author of the paper Four-Component Scattering Power Decomposition With Rotation of ...
1
vote
0answers
43 views

simple statistics of aggregated posterior data after ensemble data assimilation

I have $N$ "4-dimensional" arrays $(x,y,t,c)_{i=1:N}$ containing greenhouse gas emission data, where $(x,y)$ are the spatial coordinates $t$ is the time coordinate (discrete time points) $c$ is the ...
1
vote
1answer
27 views

Combine multiple predictions of binary outcome

I am moving from a single-model prediction of a binary outcome to an aggregate of a small number of models, for example: ...
2
votes
0answers
89 views

Combining multiple classifiers

I am trying to do a binary classification of text articles into {relevant, non-relevant}. The text articles have following features: [[article text, ...
3
votes
0answers
89 views

How to combine regression models?

Say I have three data sets of size $n$ each: $y_1$ = heights of people from the US only $y_2$ = heights of men from the whole world $y_3$ = heights of women from the whole world And I build a ...
1
vote
0answers
68 views

Ensembling with VotingClassifier

I am using VotingClassifier from sklearn.ensemble however i am puzzled with the results. Consider following algorithms: ...
1
vote
0answers
31 views

Reduction of accuracy - Bagging

Bagging is an ensemble method which uses a parallel set of classifiers and then gives consensus output. Usually bagging improves the accuracy of a classification. But if there are situations in which ...
3
votes
1answer
56 views

What happens when Bagging does not have a majority vote?

I have a question regarding the bagging technique used in ensemble learning. Let's assume I have 6 classifiers which could classify a response variable which has 3 finite categories(...
7
votes
1answer
161 views

Gradient Boosting for Linear Regression - why does it not work?

While learning about Gradient Boosting, I haven't heard about any constraints regarding the properties of a "weak classifier" that the method uses to build and ensemble model. However, I could not ...
1
vote
0answers
105 views

How to address Boosting and Bagging decreasing the classification accuracy

For my classification, I use several algorithms available in WEKA, but with limited number of features. I got some accuracy levels with the algorithms I used and I tried improving the accuracies using ...
1
vote
1answer
89 views

Accuracy reduced with Adaboost

I tried using AdaBoost for my classification which is for emotion classification. Without boosting, Random Forest algorithm gave me 42.41% of accuracy. But when I applied AdaBoost along with Random ...
7
votes
1answer
148 views

Boosting neural networks

Well recently I was working on learning boosting algorithms, such as adaboost, gradient boost, and I have known the fact that the most common used weak-learner is trees. I really want to know are ...
4
votes
0answers
87 views

Ensemble Learning: Why is Model Stacking Effective?

Recently, I've become interested in model stacking as a form of ensemble learning. In particular, I've experimented a bit with some toy datasets for regression problems. I've basically implemented ...
1
vote
0answers
31 views

Treating Categorical Variables as Continuous for Random Forest / Adaboost

What's the correct way to deal with categorical variables in packages like sklearn's RF and xgboost? Is there any cons of treating the variables are continuous? E.g. encode class A as 1, class B as ...
4
votes
2answers
109 views

Ensemble classifier methods: should we use the class probabilties or the classification itself in stacking models?

I start to work with ensembe methods these days focusing on stacking. I am wondering whether to us each models class probability ( real number in $[0,1]$) or the classifcation itself (in the binary ...
0
votes
0answers
27 views

Select the baseline methods for ensemble time-series model

I have been reading about ensemble time series models which have often been shown to perform better than single methods. I did not manage to find any reference on how to select the baseline methods. ...
1
vote
0answers
16 views

SVM ensemble with logistic regression

It is possible to average several logistic models in a ensemble using the estimated probabilities of the models. Does it make sense to calculate an ensemble based on the raw SVM score, i.e. the ...
0
votes
0answers
30 views

Generating candidates for Ensemble in machine learning

I have to compare two ensemble methods. First step for me is to create plethora of candidates using different ML algorithms. For this I took small data-sets (with 500-2000 observations) and then ...