Questions tagged [ensemble]

In machine learning, ensemble methods combine multiple algorithms to make a prediction. Bagging, boosting, and stacking, are some examples.

Filter by
Sorted by
Tagged with
0
votes
0answers
35 views

Stacking using layer-1 models predictions on test set

I am new to Data Science and have been studying the methods of stacking to find out if it can meet the following fact, but I did not find or understand evidence that it can or cannot work. Let's ...
0
votes
0answers
12 views

What does it mean when combining two features sets gives a worse result than just one of them?

I’ve created a 2 binary classifiers for lung cancer patients dead/alive 2 years after diagnosis. One is CNN trained on pathology images And the other is a RF trained on clinical data (age, sex, stage,...
0
votes
0answers
20 views

Generating ML models on the fly

I have the following scenario, given a k>=100, a set of features, a set of type of models, eg. SVR, ExtraTreeRegressor, etc: I want to generate a set of ...
1
vote
0answers
31 views

Order of features for gridsearch and model fitting

Assuming that the same columns (i.e., features) are used for hyperparameter tuning and model fitting, and ensemble models are used for modeling (e.g., Random forest or XGboost), then does the order of ...
0
votes
1answer
25 views

Is this kind of stacked ensemble method prone to over-fitting?

I am working on a stacked ensemble method. I trained three classifiers as my first-layer models and one Logistic Regression as my second layer model. I then stacked both the first-layer models and ...
4
votes
1answer
43 views

What does it mean when we say the estimators need to be independent when using Ensemble Methods in Machine Learning?

In collective learning (ensemble methods) we need the estimators to be Independent/ uncorrelated from one another. Do I understand correctly, that this means we need to draw the data samples without ...
0
votes
0answers
12 views

Finding the distribution of a quantity using ensemble

In physics, we have the concept of ensemble average, which we use a lot in statistical physics. For example, ergodic property states that the time average of a statistical quantity is equal to its ...
2
votes
1answer
30 views

Calculate minimum accuracy for a boosting algorithm

Suppose, you are working on a binary classification problem. And there are 3 models each with 70% accuracy. If you want to ensemble these models using majority voting. What will be the minimum ...
2
votes
0answers
43 views

Disadvantages of “moving window ensemble” approach?

Assuming online/incremental training is not available for a particular algorithm, and assuming that you have a stream of training data that may or may not change over time (eg log data), what are the ...
0
votes
0answers
10 views

Ensemble-based inference: How to deal with non-convergent ensemble members?

Assume that I want to use an ensemble-based sequential Bayesian inference (or swarm-based optimization) algorithm which relies on information provided by each ensemble member. For example, each sample ...
0
votes
0answers
7 views

Interpolating Regression Models Across Geographical Space

I have 50 timeseries datasets from 50 cities across the United States (1 for each city). The timeseries are of different lengths (they are daily timeseries anywhere from 3-30 years long), but they are ...
1
vote
0answers
20 views

class-specific neural networks vs overall network

Let's say I am working in a classification setting, like MNIST for instance, and imagine that I have 10 neural networks which are all slightly different and each perform very well on one unique class. ...
0
votes
0answers
21 views

Depth of trees in GBM: what does interaction.depth = 1 mean?

As far as I know, GBM does ensembling on weak learners which can be trees with one split. I have already read this post, but I could not find the answer for my question. My question is that what ...
3
votes
1answer
213 views

Is exponential loss function the only reason for AdaBoost being adaptive algorithm?

Main concept of AdaBoost is that on each iteration algorithm learns what samples were difficult to classify and increases weights of these samples, while decreasing weights of those that were easy to ...
3
votes
1answer
46 views

Train an ensemble of neural networks on different datasets, what is the best way to scale the inputs?

I'm training an ensemble of three neural networks using l-BFGS method for regression. Each neural network is trained on a sub-dataset that is randomly sampled from a large dataset. Since the sub-...
0
votes
0answers
33 views

Ensemble learning: Efficiently stacking Neural Network model

I want to use ensemble learning for a classification task. I built three models. One of them is a neural network, which takes an hour to train. I want to use model stacking. I do not have the ...
0
votes
0answers
31 views

Isolation Forest Numerical Example

I'm looking for a proper numerical example to understand Isolation Forests Algorithm correctly. I've read the paper : https://github.com/mgckind/iso_forest/blob/master/icdm08b.pdf, but I want to ...
0
votes
0answers
9 views

What's the meaning of building classifiers for each class in binary classification?

The question arises when I'm using DistributedRandomForest from the H2O package and find the ...
1
vote
0answers
112 views

How to create a test set in stacking when doing cross validation

I am using Weka to implement stacking with k-fold cross validation. As I understand, we first divide our dataset in to k folds, then we use k-1 folds for training and 1 fold for testing. This ...
1
vote
0answers
41 views

What is the most appropriate way to aggregate two identical linear models fit with different data?

Suppose I have a linear model y ~ Xb, and I split my observations into multiple X's X1, X2, X3 etc. What is the most appropriate way to aggregate the separate models y1 ~ X1b1, y2 ~ X2b2 to produce ...
2
votes
0answers
14 views

How to optimize hyperparameters in stacked model?

I was wondering whether somebody could explain how to optimize hyperparameters for the base learners and meta algorithm when stacking? In many tutorials they seem to be plucked out of thin air! ...
0
votes
2answers
223 views

Why is my stacked model worse than my base models? [closed]

I'm learning stacking and start with the approach outlined in Introduction stacking I've plotted the data: I first would like to check if my algorithm is correct (see below): So I basically ...
0
votes
0answers
12 views

ensemble learning with conv net

I extract feature with a conv net at last fully connected layer got around 85% performances on training set and 80% on test set. I use feature from CNN TEST set and I train multiple classifier (svc, ...
0
votes
0answers
13 views

How precisely do we ensemble output as in this Omni-Supervised Data Distillation paper?

After reading the paper Data Distillation: Towards Omni-Supervised Learning, I saw that after getting the results, I ensemble them before using them in order to train the student model with respect to ...
1
vote
0answers
26 views

Forecasting time series data using EEMD based SVM?

Splitting of Dataset: Dataset = Train1 + Test1 EEMD(Train1) = train1 + test1 I am forecasting on time series data("Dataset") using SVM. First I found the Intrinic Mode Function(IMF) of time series ...
1
vote
1answer
143 views

AdaBoost learning rate calculation

I saw the following in a Random Forrest calculation. My understanding of logarithms is not intuitive, I always have to look them up. It was asked: ...
0
votes
0answers
29 views

Will out of bag error always be 100% for classes of size 1 using random forests

My understanding is that given a sample data point $x_i$ the OOB prediction for this point will be calculated using only trees in the ensemble that do not contain $x_i$ in their bootstrap sample. Thus ...
0
votes
1answer
146 views

Is Stratified K Fold CV Needed when Estimator implements Balanced Class Weight?

I am working on a classification task with an imbalanced dataset. I am using Sklearn's ensemble RandomForestClassifier and set its class weight to Balanced. My question is, when I then GridSearch it, ...
1
vote
1answer
215 views

How to visualize proximity score in Random Forests

For a Random Forest, we can construct a N x N (where N is the number of data points) proximity matrix ...
1
vote
0answers
60 views

Assigning probabilities to ensemble experts (classification)

Suppose we have a set of experts which predict on a data set, and the true labels are also given. I would like to find out the probabilities for combining predictions of separate experts. So the ...
3
votes
4answers
269 views

Model ensembling - averaging of probabilities

From the BatchNorm paper, section 4.2.3, (https://arxiv.org/abs/1502.03167), The ensemble prediction was based on the arithmetic average of class probabilities predicted by the constituent ...
0
votes
0answers
26 views

Recommender system with extra variables

I would like to to create a recommendation engine that makes use of a utility matrix (user-item interactions) as well as supplementary features (user features, item features and time-based features). ...
0
votes
0answers
62 views

Modeling on entire dataset vs. Combining segmentation models trained on subsets of the same dataset

Training machine learning models on an unbalanced dataset: about 3% positive labels, and 97% negative. The modeling goal is to get as many examples as possible with 60% precision on a holdout test set ...
1
vote
0answers
27 views

Combining classifiers built on different features

I have a binary classification problem and two sets of features(for the same target variable) . Due to some reason(domain knowledge), I can't combine those two feature sets into one and then do ...
1
vote
2answers
164 views

Cross Validation versus Ensemble Learning

After performing $k$-fold cross validation to find the optimal model, and or hyperparameter choices etc, it is common to re-train your (best) proposed model on the full training set, and quote this as ...
0
votes
0answers
48 views

Using decision tree + active learning for regression?

Existing literatures that concerns using decision tree to do regression is more limited compared to its classification companion. The same also holds for research regarding active learning. I am just ...
1
vote
0answers
33 views

Does it make sense to use bagging with least square regression? if so, why?

We know that bagging can reduce the variance of the estimator, while keep the bias around the same. If we use bagging to ensemble multiple least square regressors, then are we going to reduce the ...
1
vote
2answers
80 views

Multi-View Survival Analysis

I have a data set containing various subsets of medical data about a cohort of patients. For example there are blood test results, demographics, medical examination results and a medical history among ...
2
votes
1answer
188 views

Combining Classifiers with different Precision and Recall values

Suppose I have two binary classifiers, A and B. Both are trained on the same set of data, and produce predictions on a different (but same for both classifiers) set of data. The precision for A is ...
0
votes
1answer
13 views

Ensemble learning for multiple hypothesis classes

Just to confirm if the following description falls in the category of ensemble learning. Suppose given a training set $D=\{(X,Y)\}$ we are asked to train a regressor. But now the way we do it is to ...
0
votes
1answer
210 views

Sklearn 'Seed' Not Working Properly In a Section of Code [closed]

I have written an ensemble using Scikit Learn VotingClassifier. I have set a seed in the cross validation section. However, it does not appear to 'hold'. Meaning,...
1
vote
1answer
23 views

How to design Mixture of Experts where we want only one active model at a time?

I'm trying to design a Mixture of Experts where we want only one active neural network at a time. Suppose that we have 10 experts. I want to train a MoE such that only one of the experts is active ...
2
votes
1answer
81 views

A regressor failed to learn extreme values

I am working on a regression problem using xgbclassifier (https://xgboost.readthedocs.io/en/latest/python/python_api.html) The output values range from 0 to 10 (log-normal distribution), but when I ...
1
vote
1answer
61 views

Combine mutliple predictions

This question had been asked several times in here, but I think I have something new to add. I'm interested in predicting if some specific event will happen (binary classification). I have two ...
0
votes
1answer
49 views

Gradient boosting understanding of residual picture

I recently looking at the Gradient boosting using following blog https://medium.com/mlreview/gradient-boosting-from-scratch-1e317ae4587d I try to understand the picture but I need some help For ...
0
votes
0answers
93 views

What is a good naming convention in this case (probability of assigned label being correct)?

Context: We are preparing a paper in which machine learning has been used to assign labels to data points. The machine learning algorithm is an ensemble of machines, allowing it to not only output the ...
0
votes
0answers
30 views

How to ensemble predictions from image classifier and text classifier?

I am doing multiclass classification based on images and text. I have predictions from both image classification and text. I am not sure how to combine them. Should I use probabilities as a feature to ...
2
votes
0answers
20 views

Ensemble classification model on partially overlapped datasets?

Given two partially overlapping datasets $X_1$ and $X_2$ (say past 10K hours and past 10K minutes), how could one go about creating an ensemble model of classifiers of these datasets? Standard ...
3
votes
3answers
86 views

Is it possible to use an ensemble of regression predictions to avoid issues of multicolinearity?

I am using a regression approach to make predictions using a variety of variables. However, some of my variables are pretty collinear (with a Pearson's r > 0.75), so I can't include them all in the ...
0
votes
1answer
112 views

Why AdaBoost works exactly the way it does

I understand the basic idea of AdaBoost -- when training weak classifiers, use more of the difficult examples. However, it puzzles me why I sould modify the weights the way AdaBoost does. There are, ...

1 2 3 4 5 7