Questions tagged [stacking]

Stacking is a meta-ensemble machine learning technique that trains a second-level machine learning model on the predictions from multiple machine learning models trained on the data.

Filter by
Sorted by
Tagged with
0 votes
0 answers
23 views

Outputs from models (trained on different data) as inputs to another model?

Let's say I want to detect new species of fish. I have several models, each trained to recognize a different characteristic, e.g., the speed of known fish, the size of known fish, their known shapes, ...
user avatar
  • 235
1 vote
0 answers
32 views

How to find the optimal coefficients of the two predict_proba output matrices of two different classifiers using regression and maximizing accuracy? [closed]

I am performing classification, where there are six labels and two predict_proba (predicted probabilities) matrices as outputs. These two predict_proba matrices correspond to the outputs of two ...
user avatar
0 votes
0 answers
10 views

Are stacked meta models for time series forecasting still considered forecasters?

I am working on building a meta model that is based on the principle of stacked generalisation (1). In a nutshell, this method works by using building a meta model based on the predictions of various ...
user avatar
1 vote
2 answers
39 views

How is loss propagated in stacked classifiers?

In a 2-stage stacked classifier the first model takes the input data and outputs feature vectors, which are then fed into a second model as input. The second model learns the mapping between the ...
user avatar
  • 55
1 vote
1 answer
20 views

Training twice on non-injective data

I have a large dataset of 30000 points, but most my Ys are the same while all Xs are different. Ys are from different samples, so I had means of Y for each sample and I used means alongside Xs to ...
user avatar
0 votes
0 answers
12 views

Feature Engineering for Meta-Learning?

I'm doing some stacked generalization/meta-learning. In blogs and posts, I have only seen people take the level 1 predictions and just directly use them as features for a level 2 model (no feature ...
user avatar
0 votes
0 answers
14 views

How can model averaging be done without or only a few known examples?

Model averaging has been widely used to combine multiple predictions from single models that are prone to statistical bias. As far as I know, the commonly used averaging models, including Bates-...
user avatar
2 votes
1 answer
251 views

Why is my stacking/meta-learning not outperforming the best base model?

I have a dataset of around 10,000 rows, with 500 features, response variable is binary classification. I split the features into 5 equally sized groups (based on subject matter expertise), and trained ...
user avatar
2 votes
2 answers
152 views

Why did meta-learning (or model stacking) underperform the individual base learners?

I want to use meta-learning, specifically, stacking to combine the results of two algorithms, denoted here A and B. The results of A and B correspond to the first and second columns in the dataset '...
user avatar
0 votes
1 answer
25 views

Whether can stacked generalization (stacking) further improve the performance if learner A significantly outperform B?

We know that stacking is the most popular meta-learning technique. It learns from the predictions of the base learners that learn from the training dataset. Now assuming there are two base learners, ...
user avatar
1 vote
0 answers
249 views

SHAP for stacking classifier

We are using a stacking classifier to solve a classification problem. The data feed 5 base models, the predicted probabilities of the base models feed the supervisory classifier. We would like to use ...
user avatar
0 votes
0 answers
19 views

Stacked Ensemble with Varying Weights

I have three separate models that all seek to predict the same thing per person. Each model uses a different data set with different sample sizes and then aggregates by person. For example: Model 1 is ...
user avatar
0 votes
0 answers
13 views

Combining multiple datasets vs multiple models in high dimensions

This question is related to this one and this one, but I was wondering about this topic in general. Imagine a setting where multiple datasets, representing different measurements, have been gathered. ...
user avatar
0 votes
1 answer
119 views

Ensemble Model using Stacking

I learned that building an ensemble model using stacking is done by training a meta-model on the predictions of $n$ other models in order to combine the predictions and try to enhance the performance. ...
user avatar
2 votes
0 answers
76 views

Model Stacking and tuning a meta-model - CV strategy?

I was hoping some of the more experienced ensemblers could help me with a couple of questions I have regarding stacking. The assumption is that we have a classic ...
user avatar
2 votes
2 answers
71 views

is it pointless to stack models of different data types/structure?

I am studying ensemble methods machine learning, in particular I am focusing on stacking. In stacking different models are used to get an output. Then all the outputs are 'combined' together to build ...
user avatar
2 votes
1 answer
41 views

Single or multiple model. How to know beforehand?

I am building a probability of default model based on behavioral information. The dataset is a loan portfolio, which contains 4 types of loans: mortgage, unsecured loans, car loans and credit cards. ...
user avatar
3 votes
2 answers
190 views

Why in the stacking of scikit-learn the estimators are fitted on the whole training data?

In chapter 7 of "Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow", the first step of stacking method is spliting the train data into two subsets. The first subset is used ...
user avatar
  • 33
1 vote
2 answers
218 views

StackingClassifier: Different Feature Sets

I have a few different feature sets (so, with same number of rows and the labels are the same) that I use for different ML models. In my case these sets are DataFrames. I want to use them to train a ...
user avatar
  • 11
2 votes
1 answer
180 views

Fitting Stacking Classifer when Underlying Models Use Different Feature Subsets

I’m looking to create a stacking classifier based on 3 underlying algorithms. I have already performed feature selection on each of the 3, and each returned a slightly different subset of features for ...
user avatar
  • 21
1 vote
0 answers
11 views

If you want to train a model on the predictions of another trained model, would you use the same training set for both model?

Imagine I train a model and make predictions on it, denoting the training data by x_train and the predictions by y_pred_train. Then, I want to use those predictions in a second ML model. Would you use ...
user avatar
  • 1,214
2 votes
1 answer
337 views

Score-probability obtained from Random Forest

Assume we have a classification problem with two classes $\{-1,1\}$. In my research I rather need probability, than just predicted classes. I use isotonic regression to calibrate classifier. In ...
user avatar
  • 376
2 votes
1 answer
49 views

Is it a good idea to use a linear model (like logistic regression) to generate new features for a non linear model (like random forest)? [duplicate]

The setting is a 2-class classification problem. We have too many features, some of them not very informative and with many zeros. We are thinking in ways of selecting the best features, and PCA (in ...
user avatar
1 vote
1 answer
37 views

How to detrend in a stacked model?

Let's say I have user data (x) for customers (i) in the period between 2010 and 2020 (t). I want to predict if customer i churns at time t. To do so, I have built a stacked model which looks somewhat ...
user avatar
  • 145
7 votes
1 answer
243 views

AIC model averaging when models are correlated

AIC model-averaging: In "standard" AIC model averaging we average models with weights proportional to $$w_i \propto \exp( -0.5 \times \Delta \text{AIC}_i ),$$ where $\Delta \text{AIC}_i$ is ...
user avatar
  • 23k
1 vote
0 answers
92 views

Model stacking with windowed features - info leakage?

This is similar to other questions on leakage (for example, this post), but all of my data are generated with look-back features, and nothing can be assumed to be iid. I'm curious if you think there ...
user avatar
0 votes
1 answer
214 views

StackingClassifier + RandomSearchCV: How is it dividing the folds under the hood?

I'm able to (based on the example from the accepted answer here ) set up a StackedClassifier and add RandomSearchCV to perform a quick hyperparameter search. The models/pipelines are set up like in ...
user avatar
  • 365
1 vote
2 answers
100 views

How to combine outputs from different non-machine learning prediction models?

I have a list of cancer mutations and I have run five different bioinformatics tools that predict whether a particular mutation will lead to a more aggressive form of cancer or not (0-neutral;1-...
user avatar
1 vote
0 answers
40 views

Should we train base learners on same folds when we stack different models?

I am confused about k-fold stacking. Should we train all the base learners on same folds? I mean is it ok to do the KFold split with different seeds or all base learners should trained on folds ...
user avatar
2 votes
1 answer
120 views

Model stacking, Super Learner Algorithm

I've recently started studying ensembles in ML, particularly Super Learner Algorithm. To be honest, although I have read several articles related to this topic, I am a little bit confused. I want to ...
user avatar
  • 87
6 votes
1 answer
425 views

Cross Validation in StackingClassifier Scikit-Learn

In Scikit-Learn StackingClassifier documentation it's written: Note that estimators_ are fitted on the full ...
user avatar
  • 901
0 votes
0 answers
35 views

Stacking and Ensembling methods in Data Science

Recently it seems that stacking and ensembling methods have become more popular, and using these methods can give better results than using a single algorithm. My question is: What are the reasons, ...
user avatar
  • 131
0 votes
1 answer
116 views

Why stacking use strong learner as base learner?

I am wondering why stacking uses strong learners as base learners. How to understand it from expectation and variance way, or bias and variance way?
user avatar
1 vote
0 answers
29 views

Building a set of time-dependent ensemble models: Can I stack different models at each time step?

I am trying to predict the values of a physical quantity $A(t)$ at different time steps $t$ using Machine learning. The thing is that, because my training data are time dependent, I have to make a ...
user avatar
  • 111
0 votes
0 answers
63 views

Stacking using layer-1 models predictions on test set

I am new to Data Science and have been studying the methods of stacking to find out if it can meet the following fact, but I did not find or understand evidence that it can or cannot work. Let's ...
user avatar
  • 1
1 vote
1 answer
335 views

Is this kind of stacked ensemble method prone to over-fitting?

I am working on a stacked ensemble method. I trained three classifiers as my first-layer models and one Logistic Regression as my second layer model. I then stacked both the first-layer models and ...
user avatar
  • 155
3 votes
1 answer
205 views

About the need of splitting data in stacking

I learned stacking of machine learning in a book, hands-on machine learning 2nd edition (2019). The picture was cited from hands-on machine learning 2nd edition (2019). In the above situation, ...
user avatar
  • 103
1 vote
0 answers
289 views

How to create a test set in stacking when doing cross validation

I am using Weka to implement stacking with k-fold cross validation. As I understand, we first divide our dataset in to k folds, then we use k-1 folds for training and 1 fold for testing. This ...
user avatar
  • 11
1 vote
0 answers
22 views

Should initial features be added as input of meta-learner in stacking?

I'd like to know your opinion and reference on adding initial features (i.e. the ones used to train the weak learners) to the input of the meta learner (aside of the predictions from weak learners) ...
user avatar
2 votes
0 answers
25 views

How to optimize hyperparameters in stacked model?

I was wondering whether somebody could explain how to optimize hyperparameters for the base learners and meta algorithm when stacking? In many tutorials they seem to be plucked out of thin air! ...
user avatar
  • 275
0 votes
2 answers
2k views

Why is my stacked model worse than my base models? [closed]

I'm learning stacking and start with the approach outlined in Introduction stacking I've plotted the data: I first would like to check if my algorithm is correct (see below): So I basically ...
user avatar
  • 101
0 votes
1 answer
65 views

How valid is this Stacking Model (input features to weak learners are different)?

I have a set of features with 6 of them being categorical, 1 continuous and 2 textual in type. I have to predict the labels ( 10 in number) for them. I tried applying several models and came to a ...
user avatar
0 votes
0 answers
27 views

Are there specific Machine Learning Algorithms that are more indicated for Real Time Analytics?

As the title suggests, I am wondering if there are specific ML algorithms that are more suitable for real time learning. In my case, I am working on deploying a stacking algorithm on Spark Streaming ...
user avatar
  • 634
1 vote
2 answers
207 views

I am learning to do stacking. I want to know what will the input be to level 2 classifiers [closed]

In classification/regression problems, say if we use five different base classifiers, we get 5 predictions for each example. What would be the input to the second level classifiers?
user avatar
  • 19
59 votes
7 answers
7k views

Industry vs Kaggle challenges. Is collecting more observations and having access to more variables more important than fancy modelling?

I'd hope the title is self explanatory. In Kaggle, most winners use stacking with sometimes hundreds of base models, to squeeze a few extra % of MSE, accuracy... In general, in your experience, how ...
user avatar
  • 1,214
1 vote
0 answers
57 views

Decrease the variance of ExtraTrees Classifier

I am trying to solve a machine learning problem. I am using ExtraTrees Classifier. When I am plotting the learning curves, I can see a wide gap. I need to decrease that (variance). I read about ...
user avatar
  • 23
1 vote
0 answers
332 views

Why using Out-of-fold predictions as metafeatures in stacking?

So my question is essentially the same as this one: Why do we generate out-of-fold predictions for meta-ensembling/stacking? However, I am not entirely satisfied with the answer (not detailed enough ...
user avatar
  • 1,214
7 votes
2 answers
3k views

Stacking without splitting data

I learned Stacking used in Ensemble learning. In Stacking, training data is split into two sets. The first set is used for training each model (layer-1, left figure), the second one is used for ...
user avatar
  • 181
0 votes
1 answer
526 views

Generate ensemble of classifiers based on predefined feature subsets in R using mlr

I would like to create an ensemble classifier for a dataset and use different classification models for different subsets of features (these feature subsets are predefined as the data set I am working ...
user avatar
  • 2,423
2 votes
3 answers
3k views

Why do we generate out-of-fold predictions for meta-ensembling/stacking?

Here's the guide I'm looking at: http://blog.kaggle.com/2016/12/27/a-kagglers-guide-to-model-stacking-in-practice/ Here's the relevant excerpt: The main point to take home is that we’re using the ...
user avatar