Questions tagged [ensemble]

In machine learning, ensemble methods combine multiple algorithms to make a prediction. Bagging, boosting and stacking are some examples.

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

What is the best programmatic way to find the best ensemble model in python, as to which models are best suited to which portions of the data

Generally in ensemble modelling as dataset is being segregated into multiple portions where each portion is being trained on a particular model, what is the programmatic way to determine which model ...
2
votes
1answer
21 views

What is the difference beetwen Random Forest and Random Subspace Method?

Is the only difference that the Random Forest enforces the use of decision trees as a base learner and use bootstrap sampling?
0
votes
0answers
11 views

Different types of Ensemble learning methods

I've been reading and searching information about different types of Ensemble learning methods however I am a bit confused and want to make sure my understanding is correct. Below is graph of how I ...
0
votes
1answer
28 views

Does Excess Kurtosis Signal Non-Ergodicity?

I have been reading a lot about ergodicity and the main principle behind it seems pretty simple actually. Based on my understanding, something is ergodic if the time average and the ensemble average ...
0
votes
0answers
15 views

What does it tell me when my ensemble learners (classifier trees) are just as good with only one split?

Basically new to decision trees and ensemble classifiers. Looking for guidance based on what I am seeing. My goal really isn't to use a decision tree. It is to do a simple binary (A/B) ...
1
vote
0answers
9 views

What is a reasonable number of splits (maximum) for a ensemble classifier?

I am trying to use an ensemble classifier (honing in on Matlab fitcensemble). I've also explored using a single decision tree as well as tree bagging (Matlab fitctree, TreeBagger) Simple binary (A/B) ...
0
votes
0answers
21 views

Confidence interval of ensemble prediction

I have created $N$ neural network models of the same architecture (with hyperparameters $XYZ$), each with random initializations for model weights and trained within the same workflow. These models ...
1
vote
0answers
13 views

How to combine several machine learning models trained with the same target but different sets of predictors

The book Hands-On Machine Learning with R gives an overview of model stacking, but goes on to say that the same training set shall be used for the models. The vignette of caretEnsemble also stresses ...
2
votes
1answer
34 views

Is it a good idea to use a linear model (like logistic regression) to generate new features for a non linear model (like random forest)? [duplicate]

The setting is a 2-class classification problem. We have too many features, some of them not very informative and with many zeros. We are thinking in ways of selecting the best features, and PCA (in ...
0
votes
0answers
14 views

How to tune an weighted voting ensemble method?

I am working on kidney cancer patients' data with 5 unbalanced labels. These codes are contained of Normalization, Oversampling on Feature Engineering part. A list of 9 ordinary Machine Learning ...
1
vote
0answers
26 views

in which part is used the splitting criteria in AdaBoost?

I have been reading the original article about AdaBoost and by comparing with other reading material it has come some doubts about this model. Please feel free to correct me if in any part I am wrong. ...
2
votes
1answer
25 views

Ensemble vs statistical power

In regression modelling, I've seen two schools of thought: ensemble model vs. focus on statistical power (using one model). Proponents of ensemble models (i.e., bagging) argue that: Suppose $\...
1
vote
0answers
14 views

How to sample a joint posterior given multiple models?

Consider having two models, $m_1$ and $m_2$, for a set of data $x$, each model has associated parameters $\theta_1$, for model one, and $\theta_2$ for model two, (not necessarily the same dimension). ...
1
vote
0answers
29 views

Gradient boosting doubts

I am new to the concept of Gradient Boosting and i have a few doubts related to it. It will be helpful if some one can explain them. 1) Gradient boosting is gradient descent in functional space As ...
0
votes
0answers
9 views

Improving the performance of a machine learning model on examples it has not been trained on

I'm working on generating 3D representations of objects based on images of said objects. So far I've trained 3 models with an adapted version of this code. Each model was trained on a distinct dataset ...
2
votes
0answers
48 views

Why is sklearn's CalibratedClassifierCV not labeled as an ensemble method? [closed]

I always wondered how CalibratedClassifierCV was supposed to achieve probability calibration without a dedicated calibration set (which is appealing since no data is lost for training the classifier). ...
2
votes
2answers
94 views

XGBoost Compared to Other Ensemble Methods Example

Scikit-learn has an example where it compares different "ensembles of trees" methods for classification on slices of their iris dataset. Being new to machine learning and having seen XGBoost ...
0
votes
0answers
16 views

How to choose which models to include in an ensemble

I have a variety of binary classification models (SVM, Logistic regression, LSTM, CNN, naive bayes etc.) which i want to ensemble to create a more robust model. What theories are useful here? What ...
0
votes
0answers
51 views

Why is it said that pasting using sampling without replacement?

Breiman (author of pasting) in his article written about two kinds of pasting: Rvote and Ivote. Why this and lots of other sites I can read that "When sampling is performed without replacement, ...
3
votes
0answers
36 views

Correlated random variables and ensembles (law of large numbers?)

Consider $n$ i.i.d random variables. By the law of large numbers (LLN) the sample average would converge after some time to the expected value. Let's assume the random variables are correlated. Would ...
0
votes
0answers
18 views

Is sequential testing a form of validation, a model update or a new model?

I am not sure how to call a proposed solution to combine a prediction model with a clinical test. A is a predictive model (Random forest algorithm) using information of test B that has been developed ...
0
votes
1answer
16 views

Hybrid Model: Combine two Classifiers built on different Populations / datasets

I have two classifier (A & B) built upon two distinct datasets (a & b), classifying a binary outcome (0,1). The two datasets (a & b) contain exactly the same variables, but strongly differ ...
0
votes
0answers
13 views

Basic Questions on Stacking (ensemble models)

I found a paper online "Popular Ensemble Methods: An Empirical Study" (Opitz, Maclin, 1999). Was this really the first observed use of "model stacking" (ensemble learning) in ...
1
vote
0answers
16 views

If base classifier is stable then error of ensemble is caused by bias in base classifier. Why?

I'm reading the book- Intro to Data Mining by Pang-Ning Tan. Under "Bagging" it's written: If a base classifier is stable, i.e., robust to minor perturbations in the training set, then the ...
1
vote
0answers
82 views

Should I perform nested CV with Grid Search to make my ensemble model robust?

I'm doing classification of 8 types of hand gestures with stacking models. For that I initially split the data into training and test sets. Then I used GridSerachCV ...
1
vote
1answer
30 views

Test to select best models in production

I've got four models in production and using the average of them as the served prediction. We get ground truth data immediately. I've optimized them and found the best models during my training/...
0
votes
0answers
9 views

confidence bands of average models with multiple fixed effects

I wonder whether there is a practical approach to computing sensible confidence intervals for averaged models (such as models obtained in R using the ...
0
votes
0answers
13 views

Methods for addressing model uncertainty over measures

Most of the literature on modeling uncertainty and posts on CrossValidated focus on the question of "Which subset of a set of variables should be included in the model and what functional form ...
1
vote
1answer
31 views

Which tree ensemble algorithms are the most suitable for time series forecasting (regression)?

Decision tree ensemble models are very practical for building predictive ML models. They are not strict on assumptions, can work on data without too much preprocessing, train fast and typically result ...
0
votes
0answers
10 views

Looping over algorithms results in error (ensemble, cross-validation). The function changes the shape of the predictor value

I have a dataset that I want to test with several machine learning algorithms. The features I use have a shape of (100854, 94) and the predictor value has a shape ...
1
vote
2answers
33 views

How to combine outputs from different non-machine learning prediction models?

I have a list of cancer mutations and I have run five different bioinformatics tools that predict whether a particular mutation will lead to a more aggressive form of cancer or not (0-neutral;1-...
0
votes
0answers
20 views

Is my understanding and presentation of concept of Gradient Boosting correct?

Initially the model is trained with a training set $\{x_{i}, y_{i}\}_{i=1}^{n}$ by minimizing a differentiable loss function $L(y, F(x))$, and, is initialized with a constant value, \begin{align*} F_{...
0
votes
0answers
12 views

Aggregation model in stacking with or without initial features

I am implementing a Stacking Ensemble, which works in general, for Supervised problems. Ideally, this was my idea: Train: I take the training-set and after training the N models of the ensemble, I ...
0
votes
0answers
12 views

Adaboost - Is it (really) necessary to plug sample weights into cost function?

I implemented Adaboost using the SAMME algorithm (for multiclass) with Multilayer Perceptron networks as weak learners. For the MLP, i am using ...
2
votes
1answer
119 views

How does bagging reduce variance?

I read this answer. Was still unable to understand how bagging reduces variance. Is there any other way to explain it mathematically to a newbie ? Edit Can anybody explain me this excerpt from the ...
0
votes
0answers
26 views

What is this symbol in Adaboost algorithm (SAMME)?

According to the original paper of SAMME algorithm (Adaboost for multiclass problems), it is described as follows: The general idea is very straightfoward, except for this symbol: The author didn't ...
0
votes
1answer
230 views

Ensembles for Time Series Forecasting with Python

I am building time series models in Python using the statsmodels library. Are there Python resources I can use to build ensembles of such models? (My Google searches suggest that the answer is no.)
0
votes
0answers
14 views

Is there an error in this paper loss?

I'm reading Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifiers and in section 3.1 they describe their entropy margin loss. The goal of this loss is to make the ...
1
vote
3answers
38 views

If 2 expert pieces of advice are independent then both being wrong is lower than individual advice being wrong but isn't both being right also lower?

I was watching this Coursera video on independence which ended up mentioning ensemble. I am trying to wrap my head around probability. If we ask 2 experts whether or not a startup would fail, and we ...
0
votes
0answers
23 views

Ensemble Classifiers modelling different aspects of same instance

Most of the material out there regarding Ensemble of classifiers techniques assumes that each of the classifiers that we want to combine, they are modelling the same aspect of the data. That is, they ...
2
votes
1answer
48 views

Model stacking, Super Learner Algorithm

I've recently started studying ensembles in ML, particularly Super Learner Algorithm. To be honest, although I have read several articles related to this topic, I am a little bit confused. I want to ...
1
vote
1answer
20 views

Some ambiguities about used training dataset in bagged trees

I have some ambiguity about dividing training dataset in Bagging tree. In fact I have found in this article About Decision Tree Ensembles- Bagging That : the idea is to create several subsets of data ...
1
vote
1answer
83 views

Machine Learning + Hyperparameter Tuning + Data Leakage : Is my procedure free of data leakage?

I'm trying to classify 8 types of hand gestures with EMG signals. For that I followed these steps: Split the entire data into training data and test data For training data I extracted features. Here ...
0
votes
0answers
69 views

Conditional Permutation Importance Vs Permutation Importance

After reading some papers and basic information about in: Permutation importance (PI) Conditional permutation importance (CPI) It is stated that the permutation methods it is superior to “classic” ...
0
votes
1answer
60 views

LSTM model in keras (R) with time-dependent and not time-dependent branches of inputs

I am using keras in R. I am studying 600 stations. For each station two types of information are available. The first type is ...
0
votes
0answers
3 views

Should transformed models be back-transfored before penalization? And should ensemble forecasts from these also be penalized? From the same data?

I have generated a number of regression models in an effort to predict a particular outcome. All the models are based on the same data over the same period. They include, e.g., a vector error ...
0
votes
0answers
50 views

Model adjustment during cross-validation

I have an imbalanced dataset, with the following stats: Value Count Percent 0 133412 97.62% 1 3247 2.38% I have created a ...
1
vote
1answer
27 views

Balance classifier performance (boosting ensemble)

I'm trying to build a classifier for my highly imbalanced binary data, and I'd appreciate some help on how to balance by results. The dataset has the following stats: ...
0
votes
0answers
14 views

Who first coined the term Ensemble Learning?

Who first introduced/invented the "Ensemble Techniques"? Is there any published research paper that first used that term?
1
vote
0answers
68 views

What's a word meaning “drawn from the same distribution”?

I'm comparing results from ensemble data assimilation experiments where prior ensemble members are either drawn from the same (multivariate Gaussian) distribution as a "true" variable, or ...

1
2 3 4 5
8