Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [ensemble]

In machine learning, ensemble methods combine multiple algorithms to make a prediction. Bagging, boosting, and stacking, are some examples.

2
votes
0answers
14 views

Ensemble classification model on partially overlapped datasets?

Given two partially overlapping datasets $X_1$ and $X_2$ (say past 10K hours and past 10K minutes), how could one go about creating an ensemble model of classifiers of these datasets? Standard ...
3
votes
3answers
50 views

Is it possible to use an ensemble of regression predictions to avoid issues of multicolinearity?

I am using a regression approach to make predictions using a variety of variables. However, some of my variables are pretty collinear (with a Pearson's r > 0.75), so I can't include them all in the ...
0
votes
0answers
27 views

Why AdaBoost works exactly the way it does

I understand the basic idea of AdaBoost -- when training weak classifiers, use more of the difficult examples. However, it puzzles me why I sould modify the weights the way AdaBoost does. There are, ...
0
votes
0answers
12 views

Combining Risk Scores based on Different Models

Let's say for example there are two models for different medical conditions that fall under the general category of medical conditions. For condition one, we have m features, and we built a ...
0
votes
0answers
12 views

Modeling for New vs Repeat Customer

I have a time series dataset including both new and repeat customer interactions. I noticed buyer behavior is dramatically different between the two segments, with repeat customers highly depending on ...
0
votes
0answers
10 views

Stacking Final Model Development After Cross Validation

CV (or Nested CV) are normally done to evaluate and compare different ML algorithms as part of model development and evaluation phases. Once these stages are complete, one normally develops the final ...
0
votes
0answers
11 views

Stacking Algorithms - Should Certain Algorithms Not Be Used In Conjunction

Are there any reasons why one wouldn't want to use certain algorithms together in stacking? For example, if I decided to use an SVM and a LogisticRegression classifier and were considering adding a ...
0
votes
0answers
18 views

Are there specific Machine Learning Algorithms that are more indicated for Real Time Analytics?

As the title suggests, I am wondering if there are specific ML algorithms that are more suitable for real time learning. In my case, I am working on deploying a stacking algorithm on Spark Streaming ...
0
votes
0answers
7 views

Ensemble Scalability Challenges

Are there special challenges in scalability related to having an ensemble model rather than using a single classifier?
0
votes
0answers
17 views

Combining classification models for fraud detection

i have a classification problem : fraud / non fraud. My classes are inbalanced ( 0.8% fraud rows ). I first split my data in train and test sets. Let's say I have 10 fraud and 100 non fraud rows in ...
2
votes
2answers
41 views

Is some degree of overfitting always going to occur in tree based models?

So, I am somewhat new to machine learning, and I am trying my hand at a bunch of different Kaggle datasets. In a lot of the datasets that I ended up a tree-based model on, I noticed one that all of ...
0
votes
0answers
25 views

Should I stack models or extract more features for a tiny, but hard gain in R2?

I heard that stacking models is only worth it doing it in a Kaggle competition as everyone is dealing with the same training data, and due to time limit, feature engineering only helps a little with ...
0
votes
1answer
40 views

Is it overfitting if I am using predictions from cross-validation as a level 2 feature for stacking model?

I am learning how to stack models, but I am worried if this is not a practical way to do it. I am using the full dataset and using cross_val_predict to get the ...
1
vote
0answers
36 views

Ensemble of Artificial Neural Networks

I would like to predict the probabilities of 7 classes: $$C = {c_1, ...., c_7}$$ I do not have a single model which outputs the 7 probabilities, but I have several mini-models pre-trained on different ...
4
votes
1answer
87 views

Did I understand AdaBoost correctly?

My mantra has always been that if you are not able to recreate something you haven't really understood it. In this manner I tried to implement the AdaBoost algorithm of Freund and Schapire I used one ...
2
votes
2answers
90 views

improving classification accuracy of the dataset as a whole by considering classifier distributions

Overview I'm new to machine learning so apologies if I misuse terms. I have an idea to improve my classification analysis that I feel is not terribly unique, but I can not find a reference to such a ...
0
votes
0answers
15 views

ensemble of an ensemble in Scikit Learn

I am trying to get my head around ensemble learning and need some advice. Basically, my database contains a deterministic target variable and the feature variables are all stored as probability ...
1
vote
1answer
34 views

Combination of hierarchial time series forecasts with different methods - setting weights

I am trying to forecast the the number of orders for different products of a product group. I have the time series for each product. One of the problems is that some/most time series are intermittent ...
3
votes
1answer
96 views

Why does bagging increase bias?

In machine learning, why does bagging increase bias? I've read that using less data would lead to a worse estimate of the parameters, but isn't the expected value of the parameter constant regardless ...
0
votes
0answers
11 views

Ensemble Tie-breaking Strategies

During ensemble voting, it is possible for a tie to occur when there is an equal number of votes for the majority class, or when there is no majority class because each of the individual classifiers ...
1
vote
2answers
35 views

I am learning to do stacking. I want to know what will the input be to level 2 classifiers [closed]

In classification/regression problems, say if we use five different base classifiers, we get 5 predictions for each example. What would be the input to the second level classifiers?
0
votes
0answers
167 views

Model ensemble with caretStack

I'm building a model ensemble with caretStack (package caretEnsemble). Here is a basic example : ...
0
votes
0answers
142 views

Scikit-Learn: VotingClassifier with models trained separately vs single GridSearch

I am currently training a number of separate classifiers and I want to use them to create a new Voting classifier. I currently have the code for the Voting Classifier set up as a separate GridSearch,...
0
votes
0answers
33 views

How to include new data into existing algorithm?

I have a complex ensembel algorithm X (divide data with k means that learn ensembel for each subgroup). Learning time of X is approx. 20 hours. I cannot afford to relearn algorithm for every new ...
1
vote
1answer
45 views

Why ensemble of many deep-learning models did not work?

I am trying to solve an image classification problem using DL, Keras and tensorflow. I added several layers of conv2D followed by batchnorm, pooling and dropout. I get a good accuracy ~95% with this. ...
0
votes
1answer
25 views

Why does creating training sets with replacement lead to better performance?

Pasting generally suffers from lower performance than bagging, because it training sets without replacement. Why does creating training sets with replacement lead to better performance?
0
votes
0answers
33 views

Why should each layer's child network output be close to parent network's output for variance regularizer?

I am reading up on PEA (Pseudo ensemble agreement) regularizer. specificaly in the neural networks domain. It introduces the concept of perturbing the model a little and forcing the model to make ...
2
votes
1answer
73 views

Get the number of weak learner - ebmc package of R - implementing class imbalance RusBoost on my dataset

I'm new to class imbalance and applying class imbalance technique 'RusBoost' on my dataset. I'm using ebmc package from R. I'm having difficulties to get its arguements values, as per the ...
3
votes
3answers
102 views

Why do the ensemble learners do well on regression/classification tasks?

I was watching this short video on ensemble learners, and I am confused about why they tend to do better, and how is goodness measured. If the goodness means a low mean-squared error (MSE) as usual ...
3
votes
1answer
41 views

Are ensemble learning methods for data streams restricted to online or batch learning?

Recently I'm working on some online learning algorithm (using RBF neural network ) for classification. As I read papers in this area I found there is an issue in online-learning called concept drift ...
1
vote
0answers
32 views

Why majority voting in this case works?

Suppose y is a target variable of k categories and size m. The distribution of k is unknown. And suppose there is a system that tells you your accuracy each time you make a prediction. You randomly ...
0
votes
0answers
24 views

Performing stacking classification

I have question regarding stacking classification. Just for the reference [The following kernel introduced to stacking classification method: https://www.kaggle.com/arthurtok/introduction-to-...
1
vote
0answers
23 views

Decrease the variance of ExtraTrees Classifier

I am trying to solve a machine learning problem. I am using ExtraTrees Classifier. When I am plotting the learning curves, I can see a wide gap. I need to decrease that (variance). I read about ...
1
vote
0answers
150 views

Why using Out-of-fold predictions as metafeatures in stacking?

So my question is essentially the same as this one: Why do we generate out-of-fold predictions for meta-ensembling/stacking? However, I am not entirely satisfied with the answer (not detailed enough ...
12
votes
2answers
729 views

Is there any theoretical problem with averaging regression coefficients to build a model?

I want to build a regression model that is an average of multiple OLS models, each based on a subset of the full data. The idea behind this is based on this paper. I create k folds and build k OLS ...
-1
votes
2answers
61 views

Interpreting Ensemble Models

The project I'm working on uses a lot of different variables to predict sales. The best model, in terms of mean average squared error is an ensemble Model which is a combination of a regression model ...
1
vote
1answer
48 views

Training binary classifiers with huge dataset with mostly negative examples [duplicate]

I would like to build an ensemble classifier (possibly boosting) on a huge training dataset (>> 1e7 examples) where the proportion of positive examples is around 5%. And what I am interested in are ...
1
vote
1answer
32 views

Rule based ISLE Ensemble Generation

I come through a algorithm ISLE Ensemble Generation in machine learning. The following is the steps given in Elements of Statistical Learning: But I am unable to apprehend it and implement it in ...
2
votes
2answers
447 views

Stacking without splitting data

I learned Stacking used in Ensemble learning. In Stacking, training data is split into two sets. The first set is used for training each model (layer-1, left figure), the second one is used for ...
1
vote
1answer
230 views

Combine two different multi classification models for better prediction

I have two different multi-classification models on the same dataset. Since both use the same dataset, input and output are the same. For example, given input x, model A and B output like the below. ...
0
votes
0answers
45 views

Ensemble Mean vs Ensemble; different EOF PC series

I have some gridded data, 37 years long with 40 ensemble members. I use EOF methods to obtain a leading principle component (PC) time series (in this case to calculate the NAO index from a sea level ...
2
votes
1answer
139 views

Conceptual questions on ensemble learning and Boosting methods in Matlab

The documentation on ensemble methods in Matlab explains different ensemble algorithms for classification and regression tasks. I have normalized the raw feature set and using the normalized data for ...
5
votes
1answer
1k views

How does gradient boosting calculate probability estimates?

I have been trying to understand gradient boosting reading various blogs, websites and trying to find my answer by looking through for example the XGBoost source code. However, I cannot seem to find ...
5
votes
3answers
411 views

How to describe most important features of ensemble model as list?

I have created 3 different models and output of them is a class probability in binary classification problem. Models are bit different, showing importance from different features. I have of course one ...
4
votes
1answer
1k views

hard voting, soft voting in ensemble based methods

I'm reading Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. Then I'm not able to figure out the difference between hard voting ...
0
votes
0answers
33 views

Can “stacking” ensembles be cross-validated when the test sets have only a single category (e.g. class=0)?

I'm trying to wrap my head around whether or not it is possible to use cross-validation with a stacking ensemble where each test set has only representatives from a certain class? Essentially, I ...
0
votes
1answer
303 views

Bootstrapping dataset with imbalanced classes

I am trying to build an ensemble model to classify dataset with imbalanced data, where some of classes have just a few samples. And, because of this dataset property, when I am doing re-sampling with ...
1
vote
1answer
46 views

What's the three motivations for ensemble learning?

From the Thomas Dietrich's article Ensemble Methods in Machine Learning, this 3 motivations can be concluded as: Statistical, Computational and Representational. Could anyone concluded each ...
0
votes
1answer
86 views

Generate ensemble of classifiers based on predefined feature subsets in R using mlr

I would like to create an ensemble classifier for a dataset and use different classification models for different subsets of features (these feature subsets are predefined as the data set I am working ...
2
votes
1answer
60 views

How much higher accuracy of train than test is enough to consider the model overfitted?

Considering a dataset of 920 samples with 40 features in a binary classification problem. The dataset is the heart disease dataset publicly available here. I preprocessed the dataset discarding ...