Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [ensemble]

In machine learning, ensemble methods combine multiple algorithms to make a prediction. Bagging, boosting, and stacking, are some examples.

0
votes
0answers
19 views

Recommender system with extra variables

I would like to to create a recommendation engine that makes use of a utility matrix (user-item interactions) as well as supplementary features (user features, item features and time-based features). ...
0
votes
0answers
6 views

Modeling on entire dataset vs. Combining segmentation models trained on subsets of the same dataset

Training machine learning models on an unbalanced dataset: about 3% positive labels, and 97% negative. The modeling goal is to get as many examples as possible with 60% precision on a holdout test set ...
1
vote
0answers
6 views

Combining classifiers built on different features

I have a binary classification problem and two sets of features(for the same target variable) . Due to some reason(domain knowledge), I can't combine those two feature sets into one and then do ...
1
vote
2answers
29 views

Cross Validation versus Ensemble Learning

After performing $k$-fold cross validation to find the optimal model, and or hyperparameter choices etc, it is common to re-train your (best) proposed model on the full training set, and quote this as ...
0
votes
0answers
17 views

Using decision tree + active learning for regression?

Existing literatures that concerns using decision tree to do regression is more limited compared to its classification companion. The same also holds for research regarding active learning. I am just ...
1
vote
0answers
22 views

Does it make sense to use bagging with least square regression? if so, why?

We know that bagging can reduce the variance of the estimator, while keep the bias around the same. If we use bagging to ensemble multiple least square regressors, then are we going to reduce the ...
1
vote
2answers
44 views

Multi-View Survival Analysis

I have a data set containing various subsets of medical data about a cohort of patients. For example there are blood test results, demographics, medical examination results and a medical history among ...
1
vote
1answer
35 views

Combining Classifiers with different Precision and Recall values

Suppose I have two binary classifiers, A and B. Both are trained on the same set of data, and produce predictions on a different (but same for both classifiers) set of data. The precision for A is ...
0
votes
1answer
11 views

Ensemble learning for multiple hypothesis classes

Just to confirm if the following description falls in the category of ensemble learning. Suppose given a training set $D=\{(X,Y)\}$ we are asked to train a regressor. But now the way we do it is to ...
0
votes
1answer
60 views

Sklearn 'Seed' Not Working Properly In a Section of Code [closed]

I have written an ensemble using Scikit Learn VotingClassifier. I have set a seed in the cross validation section. However, it does not appear to 'hold'. Meaning,...
1
vote
1answer
14 views

How to design Mixture of Experts where we want only one active model at a time?

I'm trying to design a Mixture of Experts where we want only one active neural network at a time. Suppose that we have 10 experts. I want to train a MoE such that only one of the experts is active ...
2
votes
1answer
34 views

A regressor failed to learn extreme values

I am working on a regression problem using xgbclassifier (https://xgboost.readthedocs.io/en/latest/python/python_api.html) The output values range from 0 to 10 (log-normal distribution), but when I ...
1
vote
1answer
39 views

Combine mutliple predictions

This question had been asked several times in here, but I think I have something new to add. I'm interested in predicting if some specific event will happen (binary classification). I have two ...
0
votes
1answer
25 views

Gradient boosting understanding of residual picture

I recently looking at the Gradient boosting using following blog https://medium.com/mlreview/gradient-boosting-from-scratch-1e317ae4587d I try to understand the picture but I need some help For ...
0
votes
0answers
18 views

What is a good naming convention in this case (probability of assigned label being correct)?

Context: We are preparing a paper in which machine learning has been used to assign labels to data points. The machine learning algorithm is an ensemble of machines, allowing it to not only output the ...
0
votes
0answers
14 views

How to ensemble predictions from image classifier and text classifier?

I am doing multiclass classification based on images and text. I have predictions from both image classification and text. I am not sure how to combine them. Should I use probabilities as a feature to ...
2
votes
0answers
17 views

Ensemble classification model on partially overlapped datasets?

Given two partially overlapping datasets $X_1$ and $X_2$ (say past 10K hours and past 10K minutes), how could one go about creating an ensemble model of classifiers of these datasets? Standard ...
3
votes
3answers
62 views

Is it possible to use an ensemble of regression predictions to avoid issues of multicolinearity?

I am using a regression approach to make predictions using a variety of variables. However, some of my variables are pretty collinear (with a Pearson's r > 0.75), so I can't include them all in the ...
0
votes
0answers
33 views

Why AdaBoost works exactly the way it does

I understand the basic idea of AdaBoost -- when training weak classifiers, use more of the difficult examples. However, it puzzles me why I sould modify the weights the way AdaBoost does. There are, ...
0
votes
0answers
27 views

Combining Risk Scores based on Different Models

Let's say for example there are two models for different medical conditions that fall under the general category of medical conditions. For condition one, we have m features, and we built a ...
0
votes
0answers
12 views

Modeling for New vs Repeat Customer

I have a time series dataset including both new and repeat customer interactions. I noticed buyer behavior is dramatically different between the two segments, with repeat customers highly depending on ...
0
votes
0answers
17 views

Stacking Final Model Development After Cross Validation

CV (or Nested CV) are normally done to evaluate and compare different ML algorithms as part of model development and evaluation phases. Once these stages are complete, one normally develops the final ...
0
votes
0answers
11 views

Stacking Algorithms - Should Certain Algorithms Not Be Used In Conjunction

Are there any reasons why one wouldn't want to use certain algorithms together in stacking? For example, if I decided to use an SVM and a LogisticRegression classifier and were considering adding a ...
0
votes
0answers
18 views

Are there specific Machine Learning Algorithms that are more indicated for Real Time Analytics?

As the title suggests, I am wondering if there are specific ML algorithms that are more suitable for real time learning. In my case, I am working on deploying a stacking algorithm on Spark Streaming ...
0
votes
0answers
8 views

Ensemble Scalability Challenges

Are there special challenges in scalability related to having an ensemble model rather than using a single classifier?
0
votes
0answers
33 views

Combining classification models for fraud detection

i have a classification problem : fraud / non fraud. My classes are inbalanced ( 0.8% fraud rows ). I first split my data in train and test sets. Let's say I have 10 fraud and 100 non fraud rows in ...
2
votes
2answers
45 views

Is some degree of overfitting always going to occur in tree based models?

So, I am somewhat new to machine learning, and I am trying my hand at a bunch of different Kaggle datasets. In a lot of the datasets that I ended up a tree-based model on, I noticed one that all of ...
0
votes
0answers
27 views

Should I stack models or extract more features for a tiny, but hard gain in R2?

I heard that stacking models is only worth it doing it in a Kaggle competition as everyone is dealing with the same training data, and due to time limit, feature engineering only helps a little with ...
0
votes
1answer
41 views

Is it overfitting if I am using predictions from cross-validation as a level 2 feature for stacking model?

I am learning how to stack models, but I am worried if this is not a practical way to do it. I am using the full dataset and using cross_val_predict to get the ...
1
vote
0answers
39 views

Ensemble of Artificial Neural Networks

I would like to predict the probabilities of 7 classes: $$C = {c_1, ...., c_7}$$ I do not have a single model which outputs the 7 probabilities, but I have several mini-models pre-trained on different ...
4
votes
1answer
110 views

Did I understand AdaBoost correctly?

My mantra has always been that if you are not able to recreate something you haven't really understood it. In this manner I tried to implement the AdaBoost algorithm of Freund and Schapire I used one ...
2
votes
2answers
115 views

improving classification accuracy of the dataset as a whole by considering classifier distributions

Overview I'm new to machine learning so apologies if I misuse terms. I have an idea to improve my classification analysis that I feel is not terribly unique, but I can not find a reference to such a ...
0
votes
0answers
15 views

ensemble of an ensemble in Scikit Learn

I am trying to get my head around ensemble learning and need some advice. Basically, my database contains a deterministic target variable and the feature variables are all stored as probability ...
1
vote
1answer
43 views

Combination of hierarchial time series forecasts with different methods - setting weights

I am trying to forecast the the number of orders for different products of a product group. I have the time series for each product. One of the problems is that some/most time series are intermittent ...
3
votes
1answer
166 views

Why does bagging increase bias?

In machine learning, why does bagging increase bias? I've read that using less data would lead to a worse estimate of the parameters, but isn't the expected value of the parameter constant regardless ...
0
votes
0answers
21 views

Ensemble Tie-breaking Strategies

During ensemble voting, it is possible for a tie to occur when there is an equal number of votes for the majority class, or when there is no majority class because each of the individual classifiers ...
1
vote
2answers
38 views

I am learning to do stacking. I want to know what will the input be to level 2 classifiers [closed]

In classification/regression problems, say if we use five different base classifiers, we get 5 predictions for each example. What would be the input to the second level classifiers?
0
votes
0answers
247 views

Model ensemble with caretStack

I'm building a model ensemble with caretStack (package caretEnsemble). Here is a basic example : ...
0
votes
0answers
230 views

Scikit-Learn: VotingClassifier with models trained separately vs single GridSearch

I am currently training a number of separate classifiers and I want to use them to create a new Voting classifier. I currently have the code for the Voting Classifier set up as a separate GridSearch,...
0
votes
0answers
33 views

How to include new data into existing algorithm?

I have a complex ensembel algorithm X (divide data with k means that learn ensembel for each subgroup). Learning time of X is approx. 20 hours. I cannot afford to relearn algorithm for every new ...
1
vote
1answer
46 views

Why ensemble of many deep-learning models did not work?

I am trying to solve an image classification problem using DL, Keras and tensorflow. I added several layers of conv2D followed by batchnorm, pooling and dropout. I get a good accuracy ~95% with this. ...
0
votes
1answer
25 views

Why does creating training sets with replacement lead to better performance?

Pasting generally suffers from lower performance than bagging, because it training sets without replacement. Why does creating training sets with replacement lead to better performance?
0
votes
0answers
34 views

Why should each layer's child network output be close to parent network's output for variance regularizer?

I am reading up on PEA (Pseudo ensemble agreement) regularizer. specificaly in the neural networks domain. It introduces the concept of perturbing the model a little and forcing the model to make ...
2
votes
1answer
79 views

Get the number of weak learner - ebmc package of R - implementing class imbalance RusBoost on my dataset

I'm new to class imbalance and applying class imbalance technique 'RusBoost' on my dataset. I'm using ebmc package from R. I'm having difficulties to get its arguements values, as per the ...
3
votes
3answers
118 views

Why do the ensemble learners do well on regression/classification tasks?

I was watching this short video on ensemble learners, and I am confused about why they tend to do better, and how is goodness measured. If the goodness means a low mean-squared error (MSE) as usual ...
3
votes
1answer
41 views

Are ensemble learning methods for data streams restricted to online or batch learning?

Recently I'm working on some online learning algorithm (using RBF neural network ) for classification. As I read papers in this area I found there is an issue in online-learning called concept drift ...
1
vote
0answers
34 views

Why majority voting in this case works?

Suppose y is a target variable of k categories and size m. The distribution of k is unknown. And suppose there is a system that tells you your accuracy each time you make a prediction. You randomly ...
0
votes
0answers
32 views

Performing stacking classification

I have question regarding stacking classification. Just for the reference [The following kernel introduced to stacking classification method: https://www.kaggle.com/arthurtok/introduction-to-...
1
vote
0answers
30 views

Decrease the variance of ExtraTrees Classifier

I am trying to solve a machine learning problem. I am using ExtraTrees Classifier. When I am plotting the learning curves, I can see a wide gap. I need to decrease that (variance). I read about ...
1
vote
0answers
180 views

Why using Out-of-fold predictions as metafeatures in stacking?

So my question is essentially the same as this one: Why do we generate out-of-fold predictions for meta-ensembling/stacking? However, I am not entirely satisfied with the answer (not detailed enough ...