Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [stacking]

Stacking is a meta-ensemble machine learning technique that trains a second-level machine learning model on the predictions from multiple machine learning models trained on the data.

0
votes
0answers
12 views

Modeling for New vs Repeat Customer

I have a time series dataset including both new and repeat customer interactions. I noticed buyer behavior is dramatically different between the two segments, with repeat customers highly depending on ...
0
votes
0answers
10 views

Stacking Final Model Development After Cross Validation

CV (or Nested CV) are normally done to evaluate and compare different ML algorithms as part of model development and evaluation phases. Once these stages are complete, one normally develops the final ...
0
votes
0answers
11 views

Stacking Algorithms - Should Certain Algorithms Not Be Used In Conjunction

Are there any reasons why one wouldn't want to use certain algorithms together in stacking? For example, if I decided to use an SVM and a LogisticRegression classifier and were considering adding a ...
0
votes
0answers
18 views

Are there specific Machine Learning Algorithms that are more indicated for Real Time Analytics?

As the title suggests, I am wondering if there are specific ML algorithms that are more suitable for real time learning. In my case, I am working on deploying a stacking algorithm on Spark Streaming ...
0
votes
0answers
25 views

Should I stack models or extract more features for a tiny, but hard gain in R2?

I heard that stacking models is only worth it doing it in a Kaggle competition as everyone is dealing with the same training data, and due to time limit, feature engineering only helps a little with ...
1
vote
2answers
35 views

I am learning to do stacking. I want to know what will the input be to level 2 classifiers [closed]

In classification/regression problems, say if we use five different base classifiers, we get 5 predictions for each example. What would be the input to the second level classifiers?
1
vote
0answers
49 views

How people use Stacking method in the real-world problems? [closed]

I know that stacking is a very strong method when you using it in machine learning competition(Like Kaggle). But in real life situations do people use this for modeling very often? I heard that ...
54
votes
7answers
7k views

Industry vs Kaggle challenges. Is collecting more observations and having access to more variables more important than fancy modelling?

I'd hope the title is self explanatory. In Kaggle, most winners use stacking with sometimes hundreds of base models, to squeeze a few extra % of MSE, accuracy... In general, in your experience, how ...
0
votes
0answers
24 views

Performing stacking classification

I have question regarding stacking classification. Just for the reference [The following kernel introduced to stacking classification method: https://www.kaggle.com/arthurtok/introduction-to-...
1
vote
0answers
23 views

Decrease the variance of ExtraTrees Classifier

I am trying to solve a machine learning problem. I am using ExtraTrees Classifier. When I am plotting the learning curves, I can see a wide gap. I need to decrease that (variance). I read about ...
1
vote
0answers
150 views

Why using Out-of-fold predictions as metafeatures in stacking?

So my question is essentially the same as this one: Why do we generate out-of-fold predictions for meta-ensembling/stacking? However, I am not entirely satisfied with the answer (not detailed enough ...
2
votes
2answers
447 views

Stacking without splitting data

I learned Stacking used in Ensemble learning. In Stacking, training data is split into two sets. The first set is used for training each model (layer-1, left figure), the second one is used for ...
0
votes
0answers
33 views

Can “stacking” ensembles be cross-validated when the test sets have only a single category (e.g. class=0)?

I'm trying to wrap my head around whether or not it is possible to use cross-validation with a stacking ensemble where each test set has only representatives from a certain class? Essentially, I ...
0
votes
1answer
86 views

Generate ensemble of classifiers based on predefined feature subsets in R using mlr

I would like to create an ensemble classifier for a dataset and use different classification models for different subsets of features (these feature subsets are predefined as the data set I am working ...
0
votes
0answers
36 views

Training multiple deep models for image classification

I'm looking to classify images of cars or car parts into these 7 labels: left front right front left rear right rear dashboard interior odometer Where the first 4 labels describe the angle at which ...
0
votes
0answers
139 views

Recalculating probabilities instead of decision threshold

First I fit multiclass xgboost model using multi:softprob objective. Then based on some criteria I choose a threshold between class #4 and class #5. E. g. if ...
1
vote
1answer
777 views

Why do we generate out-of-fold predictions for meta-ensembling/stacking?

Here's the guide I'm looking at: http://blog.kaggle.com/2016/12/27/a-kagglers-guide-to-model-stacking-in-practice/ Here's the relevant excerpt: The main point to take home is that we’re using the ...
2
votes
1answer
328 views

Combining bagging and stacking, with and without clusters and heteroskedasticity

Question 1: Start with the classing case of bagging, say in random forest. Fit $B$ trees to bootstrap samples of the data. Average the predictions of the $B$ trees to form a final prediction. ...
1
vote
1answer
79 views

Stacking or Voting - Multiple feature sets extracted with different parameters

I am extracting features from time series data using different parameters and then creating a SINGLE feature based data set with all features to perform classification. If I wanted to create separate ...
1
vote
1answer
455 views

Model stacking, what is the input of meta classifier?

I know that by stacking different models among which there has a low correlation can boost the performance of on single model. And I found a picture In step 7, the $h_j(x_i)$ in new data $x_i^{'}=\{...
1
vote
0answers
57 views

Stacking Lasso models

let say I have M subsets of independent variables and I want to use stack learner to predict dependent variable y. for each subset I use lasso method to get meta features (predictions). I have 2 ...
0
votes
0answers
210 views

Stacking, How can I check the correlation of models?

In ensemble learning, model stacking is a good way to improve the performance of a single model. However, models chosen to be stacked must have least correlation in order to exploit the performance of ...
0
votes
0answers
69 views

a cross validation funciton in kaggle kernel codes

I am new to Data mining and recently read a kaggle kernel:https://www.kaggle.com/karakos/fork-introduction-to-ensembling-stacking there is function below: ...
3
votes
1answer
147 views

Can we stack the strong learners? [duplicate]

Stacking is done with combining all the weak learners. What will happen if we do it with strong learners? A case of overfitting?
3
votes
3answers
133 views

Models combinations

My goal is time series forecasting. I have created a number of models to make predictions. I know that forecast quality can be improved by combining predictions from different models(linear ...
4
votes
1answer
4k views

How to stack machine learning models in R

I am new to machine learning and R. I know that there is an R package called caretEnsemble, which could conveniently stack the models in R. However, this package looks has some problems when deals ...
1
vote
1answer
119 views

Using one ml models output to choose another models input

I'm dealing with a low event rate problem (e.g. credit card fraud). I've balanced my data with SMOTE, and ran a neural net model (cross validated with recall as the measure). However my precision (...
1
vote
0answers
32 views

Using different data transformation in ensembles

I have trained few models using sklearn and python. However I have scaled the data for Support Vector Machines - standarized and for Neural Network - Scaled [0-1] since it gives me better results and ...
2
votes
0answers
248 views

Why StackingRegressor doesn't catch the trend?

I just reviewed very good example of fitting StackingRegressor from mlxtend package. ...
5
votes
2answers
2k views

Model Stacking - Gives poor performance

I'm trying model stacking in a kaggle competition. However, what the competition is trying to do is irrelevant. I think my approach of doing model stacking is not correct. I have 4 different models: ...
1
vote
2answers
158 views

how do you stack models with 100% training accuracy?

Suppose I have several models, one of them $M$ has a 100% training accuracy. So regardless of how a stack the models, the stacked model is just M. e.g. If I use a linear model to stack them, then the ...
0
votes
1answer
507 views

Ensemble Decision Trees and Gradient Boosted Decision Trees

I see people often ensemble Gradient Boosted Decision Trees and Random Forests together. Does it make sense to ensemble a Decision Tree and a GBDT together? Isn't this DT already a part of GBDT?
1
vote
0answers
68 views

Stacking sensitivity analysis

I conducted stacking of three algorithms (NN, J48 and BN) with logistic regression as the meta-classifier. I am interested in doing a sensitivity analysis so I am able to rank the predictors and ...
1
vote
1answer
55 views

Training separate models for sets of features?

I'm wondering if it is sensible to train different models for different sets of features (I do not mean one model for each feature). Say I have 4 features - 2 are nominal while the other two are 2 ...
-1
votes
1answer
494 views

Stacking Ensemble Meta Learning Components

I have been learning about Ensemble Algorithms. Reading this paper, I want to know about the meta classifiers that work well with Stacking Ensemble Learning Algo, since this work just talks about ...
0
votes
1answer
53 views

How do you calculate the covariance “stack up” of relative measurements?

Say that you have a measurement of $x_1$ relative to some globally fixed datum and a covariance for it, $\sigma_1$. If you have another measurement, $x_2$, taken relative to $x_1$ with its own ...
3
votes
1answer
741 views

How to properly do stacking/meta ensembling with cross validation

How do people use stacking or meta ensembling with cross validation in practice and in machine learning competitions like on Kaggle? Here are two approaches I've seen (but maybe neither is correct?) ...
0
votes
1answer
119 views

Does my classification model suffer from information leak?

I'm trying to solve a recommendation problem (recommending items to users). I have a dataset of triplets (user, item, reaction), where reaction is either 0 or 1 (...
1
vote
1answer
912 views

Stacking models which trained by different features in a data set for a classification problem

Normally, the first layer models in stacking and bledning method are trained by all features in a data set. However, what will be the performance of a big model based on first layer models trained by ...
5
votes
1answer
845 views

Stacking: Do more base classifiers always improve accuracy?

When using stacking, can accuracy always be improved by adding more base classifiers, types of base classifiers, and features?
2
votes
0answers
70 views

Stacking/Blending Predictive Models with More than Two Outcomes

I've been experimenting with stacking predictive models recently. I've mainly been focused on looking at making meta-models based off of predictive probabilities of smaller models while implementing ...
4
votes
1answer
920 views

Isn't stacking models a direct approach to overfitting?

With help by the discussions here I successfully trained various models for classification. As an example say I trained a stochastic gradient boosted model (gbm) and an extreme gradient boosted tree ...
0
votes
1answer
865 views

Stacking GBT using Logistic Regression

1) I build Gradient Boosted Tree Model in h2o and now i have the POJO. 2) I extracted the weight for each tree of GBT for my population 3) I used the extracted weight to train a logistic regression ...
2
votes
1answer
561 views

How to evaluate stacking ensemble model vs. other models with 10-fold cross-validation?

I've been comparing various predictive models for both continuous and binary outcomes for a health care model. So far 10-fold cross-validation has been useful: training models on 9/10 of the analysis ...
1
vote
0answers
130 views

Model stacking, should the folds in the training set be the same?

I am stacking various models (Gradient Boosting Machines, Random Forests, Linear Regressions) using a k-fold cross validation for the train set $X_{train}$, therefore obtaining out-of-sample ...
3
votes
0answers
183 views

stacking complex models that are prone to overfitting

I am working on a CNN (convolutional neural net) model to classify certain songbirds. I am using one CNN to classsify images of the bird, and one CNN to classify audio sounds of the birds. I would ...
6
votes
1answer
6k views

stacking and blending of regression models

I am self-studying blending and stacking, and am especially interested in this in the context of regression models. I have been reading a number of the stacking, blending and bagging links posted on ...
33
votes
2answers
3k views

Is this the state of art regression methodology?

I've been following Kaggle competitions for a long time and I come to realize that many winning strategies involve using at least one of the "big threes": bagging, boosting and stacking. For ...
8
votes
1answer
706 views

Ensemble Learning: Why is Model Stacking Effective?

Recently, I've become interested in model stacking as a form of ensemble learning. In particular, I've experimented a bit with some toy datasets for regression problems. I've basically implemented ...
1
vote
0answers
346 views

Stacking ensembles to improve prediction

I recently read this blog and it has many ideas for ensembling various models. I created three models for my training data, random forest model, SVM model and a KNN model. However when I use linear ...