Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [ensemble]

In machine learning, ensemble methods combine multiple algorithms to make a prediction. Bagging, boosting, and stacking, are some examples.

0
votes
0answers
8 views

ensemble learning with conv net

I extract feature with a conv net at last fully connected layer got around 85% performances on training set and 80% on test set. I use feature from CNN TEST set and I train multiple classifier (svc, ...
0
votes
0answers
11 views

How precisely do we ensemble output as in this Omni-Supervised Data Distillation paper?

After reading the paper Data Distillation: Towards Omni-Supervised Learning, I saw that after getting the results, I ensemble them before using them in order to train the student model with respect to ...
1
vote
0answers
16 views

Forecasting time series data using EEMD based SVM?

Splitting of Dataset: Dataset = Train1 + Test1 EEMD(Train1) = train1 + test1 I am forecasting on time series data("Dataset") using SVM. First I found the Intrinic Mode Function(IMF) of time series ...
0
votes
1answer
49 views

Learning rate constant in Random Forrest calculation

I saw the following in a Random Forrest calculation. My understanding of logarithms is not intuitive, I always have to look them up. It was asked: ...
0
votes
0answers
27 views

Will out of bag error always be 100% for classes of size 1 using random forests

My understanding is that given a sample data point $x_i$ the OOB prediction for this point will be calculated using only trees in the ensemble that do not contain $x_i$ in their bootstrap sample. Thus ...
0
votes
1answer
12 views

Is Stratified K Fold CV Needed when Estimator implements Balanced Class Weight?

I am working on a classification task with an imbalanced dataset. I am using Sklearn's ensemble RandomForestClassifier and set its class weight to Balanced. My question is, when I then GridSearch it, ...
1
vote
1answer
54 views

How to visualize proximity score in Random Forests

For a Random Forest, we can construct a N x N (where N is the number of data points) proximity matrix ...
0
votes
0answers
22 views

Assigning probabilities to ensemble experts (classification)

Suppose we have a set of experts which predict on a data set, and the true labels are also given. I would like to find out the probabilities for combining predictions of separate experts. So the ...
2
votes
4answers
179 views

Model ensembling - averaging of probabilities

From the BatchNorm paper, section 4.2.3, (https://arxiv.org/abs/1502.03167), The ensemble prediction was based on the arithmetic average of class probabilities predicted by the constituent ...
0
votes
0answers
23 views

Recommender system with extra variables

I would like to to create a recommendation engine that makes use of a utility matrix (user-item interactions) as well as supplementary features (user features, item features and time-based features). ...
0
votes
0answers
8 views

Modeling on entire dataset vs. Combining segmentation models trained on subsets of the same dataset

Training machine learning models on an unbalanced dataset: about 3% positive labels, and 97% negative. The modeling goal is to get as many examples as possible with 60% precision on a holdout test set ...
1
vote
0answers
10 views

Combining classifiers built on different features

I have a binary classification problem and two sets of features(for the same target variable) . Due to some reason(domain knowledge), I can't combine those two feature sets into one and then do ...
1
vote
2answers
41 views

Cross Validation versus Ensemble Learning

After performing $k$-fold cross validation to find the optimal model, and or hyperparameter choices etc, it is common to re-train your (best) proposed model on the full training set, and quote this as ...
0
votes
0answers
24 views

Using decision tree + active learning for regression?

Existing literatures that concerns using decision tree to do regression is more limited compared to its classification companion. The same also holds for research regarding active learning. I am just ...
1
vote
0answers
27 views

Does it make sense to use bagging with least square regression? if so, why?

We know that bagging can reduce the variance of the estimator, while keep the bias around the same. If we use bagging to ensemble multiple least square regressors, then are we going to reduce the ...
1
vote
2answers
55 views

Multi-View Survival Analysis

I have a data set containing various subsets of medical data about a cohort of patients. For example there are blood test results, demographics, medical examination results and a medical history among ...
1
vote
1answer
46 views

Combining Classifiers with different Precision and Recall values

Suppose I have two binary classifiers, A and B. Both are trained on the same set of data, and produce predictions on a different (but same for both classifiers) set of data. The precision for A is ...
0
votes
1answer
11 views

Ensemble learning for multiple hypothesis classes

Just to confirm if the following description falls in the category of ensemble learning. Suppose given a training set $D=\{(X,Y)\}$ we are asked to train a regressor. But now the way we do it is to ...
0
votes
1answer
77 views

Sklearn 'Seed' Not Working Properly In a Section of Code [closed]

I have written an ensemble using Scikit Learn VotingClassifier. I have set a seed in the cross validation section. However, it does not appear to 'hold'. Meaning,...
1
vote
1answer
14 views

How to design Mixture of Experts where we want only one active model at a time?

I'm trying to design a Mixture of Experts where we want only one active neural network at a time. Suppose that we have 10 experts. I want to train a MoE such that only one of the experts is active ...
2
votes
1answer
39 views

A regressor failed to learn extreme values

I am working on a regression problem using xgbclassifier (https://xgboost.readthedocs.io/en/latest/python/python_api.html) The output values range from 0 to 10 (log-normal distribution), but when I ...
1
vote
1answer
43 views

Combine mutliple predictions

This question had been asked several times in here, but I think I have something new to add. I'm interested in predicting if some specific event will happen (binary classification). I have two ...
0
votes
1answer
27 views

Gradient boosting understanding of residual picture

I recently looking at the Gradient boosting using following blog https://medium.com/mlreview/gradient-boosting-from-scratch-1e317ae4587d I try to understand the picture but I need some help For ...
0
votes
0answers
34 views

What is a good naming convention in this case (probability of assigned label being correct)?

Context: We are preparing a paper in which machine learning has been used to assign labels to data points. The machine learning algorithm is an ensemble of machines, allowing it to not only output the ...
0
votes
0answers
17 views

How to ensemble predictions from image classifier and text classifier?

I am doing multiclass classification based on images and text. I have predictions from both image classification and text. I am not sure how to combine them. Should I use probabilities as a feature to ...
2
votes
0answers
18 views

Ensemble classification model on partially overlapped datasets?

Given two partially overlapping datasets $X_1$ and $X_2$ (say past 10K hours and past 10K minutes), how could one go about creating an ensemble model of classifiers of these datasets? Standard ...
3
votes
3answers
68 views

Is it possible to use an ensemble of regression predictions to avoid issues of multicolinearity?

I am using a regression approach to make predictions using a variety of variables. However, some of my variables are pretty collinear (with a Pearson's r > 0.75), so I can't include them all in the ...
0
votes
1answer
66 views

Why AdaBoost works exactly the way it does

I understand the basic idea of AdaBoost -- when training weak classifiers, use more of the difficult examples. However, it puzzles me why I sould modify the weights the way AdaBoost does. There are, ...
0
votes
0answers
48 views

Combining Risk Scores based on Different Models

Let's say for example there are two models for different medical conditions that fall under the general category of medical conditions. For condition one, we have m features, and we built a ...
0
votes
0answers
12 views

Modeling for New vs Repeat Customer

I have a time series dataset including both new and repeat customer interactions. I noticed buyer behavior is dramatically different between the two segments, with repeat customers highly depending on ...
0
votes
0answers
28 views

Stacking Final Model Development After Cross Validation

CV (or Nested CV) are normally done to evaluate and compare different ML algorithms as part of model development and evaluation phases. Once these stages are complete, one normally develops the final ...
0
votes
0answers
11 views

Stacking Algorithms - Should Certain Algorithms Not Be Used In Conjunction

Are there any reasons why one wouldn't want to use certain algorithms together in stacking? For example, if I decided to use an SVM and a LogisticRegression classifier and were considering adding a ...
0
votes
0answers
19 views

Are there specific Machine Learning Algorithms that are more indicated for Real Time Analytics?

As the title suggests, I am wondering if there are specific ML algorithms that are more suitable for real time learning. In my case, I am working on deploying a stacking algorithm on Spark Streaming ...
0
votes
0answers
8 views

Ensemble Scalability Challenges

Are there special challenges in scalability related to having an ensemble model rather than using a single classifier?
0
votes
0answers
50 views

Combining classification models for fraud detection

i have a classification problem : fraud / non fraud. My classes are inbalanced ( 0.8% fraud rows ). I first split my data in train and test sets. Let's say I have 10 fraud and 100 non fraud rows in ...
2
votes
2answers
47 views

Is some degree of overfitting always going to occur in tree based models?

So, I am somewhat new to machine learning, and I am trying my hand at a bunch of different Kaggle datasets. In a lot of the datasets that I ended up a tree-based model on, I noticed one that all of ...
0
votes
0answers
29 views

Should I stack models or extract more features for a tiny, but hard gain in R2?

I heard that stacking models is only worth it doing it in a Kaggle competition as everyone is dealing with the same training data, and due to time limit, feature engineering only helps a little with ...
0
votes
1answer
44 views

Is it overfitting if I am using predictions from cross-validation as a level 2 feature for stacking model?

I am learning how to stack models, but I am worried if this is not a practical way to do it. I am using the full dataset and using cross_val_predict to get the ...
1
vote
0answers
40 views

Ensemble of Artificial Neural Networks

I would like to predict the probabilities of 7 classes: $$C = {c_1, ...., c_7}$$ I do not have a single model which outputs the 7 probabilities, but I have several mini-models pre-trained on different ...
4
votes
1answer
133 views

Did I understand AdaBoost correctly?

My mantra has always been that if you are not able to recreate something you haven't really understood it. In this manner I tried to implement the AdaBoost algorithm of Freund and Schapire I used one ...
2
votes
2answers
130 views

improving classification accuracy of the dataset as a whole by considering classifier distributions

Overview I'm new to machine learning so apologies if I misuse terms. I have an idea to improve my classification analysis that I feel is not terribly unique, but I can not find a reference to such a ...
0
votes
0answers
16 views

ensemble of an ensemble in Scikit Learn

I am trying to get my head around ensemble learning and need some advice. Basically, my database contains a deterministic target variable and the feature variables are all stored as probability ...
1
vote
1answer
52 views

Combination of hierarchial time series forecasts with different methods - setting weights

I am trying to forecast the the number of orders for different products of a product group. I have the time series for each product. One of the problems is that some/most time series are intermittent ...
3
votes
1answer
262 views

Why does bagging increase bias?

In machine learning, why does bagging increase bias? I've read that using less data would lead to a worse estimate of the parameters, but isn't the expected value of the parameter constant regardless ...
0
votes
0answers
34 views

Ensemble Tie-breaking Strategies

During ensemble voting, it is possible for a tie to occur when there is an equal number of votes for the majority class, or when there is no majority class because each of the individual classifiers ...
1
vote
2answers
46 views

I am learning to do stacking. I want to know what will the input be to level 2 classifiers [closed]

In classification/regression problems, say if we use five different base classifiers, we get 5 predictions for each example. What would be the input to the second level classifiers?
1
vote
0answers
293 views

Model ensemble with caretStack

I'm building a model ensemble with caretStack (package caretEnsemble). Here is a basic example : ...
0
votes
0answers
264 views

Scikit-Learn: VotingClassifier with models trained separately vs single GridSearch

I am currently training a number of separate classifiers and I want to use them to create a new Voting classifier. I currently have the code for the Voting Classifier set up as a separate GridSearch,...
0
votes
0answers
34 views

How to include new data into existing algorithm?

I have a complex ensembel algorithm X (divide data with k means that learn ensembel for each subgroup). Learning time of X is approx. 20 hours. I cannot afford to relearn algorithm for every new ...
1
vote
1answer
48 views

Why ensemble of many deep-learning models did not work?

I am trying to solve an image classification problem using DL, Keras and tensorflow. I added several layers of conv2D followed by batchnorm, pooling and dropout. I get a good accuracy ~95% with this. ...