Questions tagged [ensemble-learning]

In machine learning, ensemble methods combine multiple algorithms to make a prediction. Bagging, boosting and stacking are some examples.

Filter by
Sorted by
Tagged with
0
votes
0answers
12 views

NLP Classification over Taxonomy [closed]

I have a massive pool of data where each row maps a user-entered description to a code that classifies the mission that the description describes. I am trying to develop a model that at least helps ...
1
vote
1answer
67 views

Does Wikipedia explain gradient boosting in wrong way?

Wikipedia's geral Gradient Boosting is: Friedman's Gradient Boosting is: Why wikipedia's gradient boosting fit h_m through pseudo-residuals while friedman uses line 4 to fit h? My question is not ...
2
votes
2answers
42 views

Does Adaboost ensemble use bootstrapping?

I am reading about boosting methods in the book Elements of statistical learning. In page 339 they describe the Ada boost algorithm as I understand the general idea behind it: Give more weight to ...
2
votes
0answers
8 views

Gradient Boosting algorithm

What is beta for? Why not just fit h_m (get a_m) without this beta? since h_m is an estimator for pseudo-residuals
0
votes
1answer
29 views

How to interpret the direct comparison of Continuous Rank Probability Score (CRPS) and Mean Absolute Error (MAE)?

Say I have a trained Random Forest (RF) consisted of $m$ decision trees and I am interested to estimate $y$ from $t_1$ to $t_n$. The good thing about RF is that I have an ensemble of estimators and a ...
0
votes
0answers
9 views

How to build an ensemble for different set of features?

I have a dataset in which part of the features have more data than the other part, and for avoiding to build a full data set with a small amount of data, I'd like to build two models, each one ...
1
vote
1answer
19 views

Is multicollinearity ever an issue in ensemble learning?

Suppose I have two models, A and B, and suppose B takes the output of A as one of its features. Now suppose that both models use at least some of the same features. Is there a potential ...
0
votes
0answers
14 views

Methods to reduce regression underestimate and overestimate

I'm new to a project and need to reduce the underestimate & overestimate cases in a regression problem. So far haven't gained enough domain knowledge. Underestimate is less tolerable than ...
0
votes
1answer
28 views

Is it necessary to retrain a random forest instead of removing trees when comparing accuracy between different numbers of trees?

I have a train data set and a validation set using which I wish to optimize the hyperparameter that is the number of trees in a binary classification random forest (scikit-learn). (As Sycorax ...
0
votes
0answers
11 views

What is the best programmatic way to find the best ensemble model in python, as to which models are best suited to which portions of the data

Generally in ensemble modelling as dataset is being segregated into multiple portions where each portion is being trained on a particular model, what is the programmatic way to determine which model ...
2
votes
1answer
26 views

What is the difference beetwen Random Forest and Random Subspace Method?

Is the only difference that the Random Forest enforces the use of decision trees as a base learner and use bootstrap sampling?
0
votes
0answers
14 views

Different types of Ensemble learning methods

I've been reading and searching information about different types of Ensemble learning methods however I am a bit confused and want to make sure my understanding is correct. Below is graph of how I ...
0
votes
1answer
29 views

Does Excess Kurtosis Signal Non-Ergodicity?

I have been reading a lot about ergodicity and the main principle behind it seems pretty simple actually. Based on my understanding, something is ergodic if the time average and the ensemble average ...
0
votes
0answers
15 views

What does it tell me when my ensemble learners (classifier trees) are just as good with only one split?

Basically new to decision trees and ensemble classifiers. Looking for guidance based on what I am seeing. My goal really isn't to use a decision tree. It is to do a simple binary (A/B) ...
1
vote
0answers
22 views

What is a reasonable number of splits (maximum) for a ensemble classifier?

I am trying to use an ensemble classifier (honing in on Matlab fitcensemble). I've also explored using a single decision tree as well as tree bagging (Matlab fitctree, TreeBagger) Simple binary (A/B) ...
0
votes
0answers
65 views

Confidence interval of ensemble prediction

I have created $N$ neural network models of the same architecture (with hyperparameters $XYZ$), each with random initializations for model weights and trained within the same workflow. These models ...
1
vote
0answers
15 views

How to combine several machine learning models trained with the same target but different sets of predictors

The book Hands-On Machine Learning with R gives an overview of model stacking, but goes on to say that the same training set shall be used for the models. The vignette of caretEnsemble also stresses ...
2
votes
1answer
37 views

Is it a good idea to use a linear model (like logistic regression) to generate new features for a non linear model (like random forest)? [duplicate]

The setting is a 2-class classification problem. We have too many features, some of them not very informative and with many zeros. We are thinking in ways of selecting the best features, and PCA (in ...
0
votes
0answers
17 views

How to tune an weighted voting ensemble method?

I am working on kidney cancer patients' data with 5 unbalanced labels. These codes are contained of Normalization, Oversampling on Feature Engineering part. A list of 9 ordinary Machine Learning ...
2
votes
1answer
43 views

in which part is used the splitting criteria in AdaBoost?

I have been reading the original article about AdaBoost and by comparing with other reading material it has come some doubts about this model. Please feel free to correct me if in any part I am wrong. ...
2
votes
1answer
27 views

Ensemble vs statistical power

In regression modelling, I've seen two schools of thought: ensemble model vs. focus on statistical power (using one model). Proponents of ensemble models (i.e., bagging) argue that: Suppose $\...
1
vote
0answers
16 views

How to sample a joint posterior given multiple models?

Consider having two models, $m_1$ and $m_2$, for a set of data $x$, each model has associated parameters $\theta_1$, for model one, and $\theta_2$ for model two, (not necessarily the same dimension). ...
1
vote
0answers
30 views

Gradient boosting doubts

I am new to the concept of Gradient Boosting and i have a few doubts related to it. It will be helpful if some one can explain them. 1) Gradient boosting is gradient descent in functional space As ...
0
votes
0answers
9 views

Improving the performance of a machine learning model on examples it has not been trained on

I'm working on generating 3D representations of objects based on images of said objects. So far I've trained 3 models with an adapted version of this code. Each model was trained on a distinct dataset ...
2
votes
0answers
101 views

Why is sklearn's CalibratedClassifierCV not labeled as an ensemble method? [closed]

I always wondered how CalibratedClassifierCV was supposed to achieve probability calibration without a dedicated calibration set (which is appealing since no data is lost for training the classifier). ...
2
votes
2answers
103 views

XGBoost Compared to Other Ensemble Methods Example

Scikit-learn has an example where it compares different "ensembles of trees" methods for classification on slices of their iris dataset. Being new to machine learning and having seen XGBoost ...
0
votes
0answers
18 views

How to choose which models to include in an ensemble

I have a variety of binary classification models (SVM, Logistic regression, LSTM, CNN, naive bayes etc.) which i want to ensemble to create a more robust model. What theories are useful here? What ...
0
votes
0answers
53 views

Why is it said that pasting using sampling without replacement?

Breiman (author of pasting) in his article written about two kinds of pasting: Rvote and Ivote. Why this and lots of other sites I can read that "When sampling is performed without replacement, ...
3
votes
0answers
38 views

Correlated random variables and ensembles (law of large numbers?)

Consider $n$ i.i.d random variables. By the law of large numbers (LLN) the sample average would converge after some time to the expected value. Let's assume the random variables are correlated. Would ...
0
votes
0answers
42 views

Is sequential testing a form of validation, a model update or a new model?

I am not sure how to call an approach to combine a prediction model with a clinical imaging test. Model A is a prediction model (eg, Random forest algorithm or Logistic regression) using quantitative ...
0
votes
1answer
21 views

Hybrid Model: Combine two Classifiers built on different Populations / datasets

I have two classifier (A & B) built upon two distinct datasets (a & b), classifying a binary outcome (0,1). The two datasets (a & b) contain exactly the same variables, but strongly differ ...
0
votes
0answers
16 views

Basic Questions on Stacking (ensemble models)

I found a paper online "Popular Ensemble Methods: An Empirical Study" (Opitz, Maclin, 1999). Was this really the first observed use of "model stacking" (ensemble learning) in ...
1
vote
0answers
17 views

If base classifier is stable then error of ensemble is caused by bias in base classifier. Why?

I'm reading the book- Intro to Data Mining by Pang-Ning Tan. Under "Bagging" it's written: If a base classifier is stable, i.e., robust to minor perturbations in the training set, then the ...
1
vote
0answers
86 views

Should I perform nested CV with Grid Search to make my ensemble model robust?

I'm doing classification of 8 types of hand gestures with stacking models. For that I initially split the data into training and test sets. Then I used GridSerachCV ...
1
vote
1answer
30 views

Test to select best models in production

I've got four models in production and using the average of them as the served prediction. We get ground truth data immediately. I've optimized them and found the best models during my training/...
0
votes
0answers
15 views

confidence bands of average models with multiple fixed effects

I wonder whether there is a practical approach to computing sensible confidence intervals for averaged models (such as models obtained in R using the ...
0
votes
0answers
14 views

Methods for addressing model uncertainty over measures

Most of the literature on modeling uncertainty and posts on CrossValidated focus on the question of "Which subset of a set of variables should be included in the model and what functional form ...
1
vote
1answer
45 views

Which tree ensemble algorithms are the most suitable for time series forecasting (regression)?

Decision tree ensemble models are very practical for building predictive ML models. They are not strict on assumptions, can work on data without too much preprocessing, train fast and typically result ...
0
votes
0answers
13 views

Looping over algorithms results in error (ensemble, cross-validation). The function changes the shape of the predictor value

I have a dataset that I want to test with several machine learning algorithms. The features I use have a shape of (100854, 94) and the predictor value has a shape ...
1
vote
2answers
42 views

How to combine outputs from different non-machine learning prediction models?

I have a list of cancer mutations and I have run five different bioinformatics tools that predict whether a particular mutation will lead to a more aggressive form of cancer or not (0-neutral;1-...
0
votes
0answers
20 views

Is my understanding and presentation of concept of Gradient Boosting correct?

Initially the model is trained with a training set $\{x_{i}, y_{i}\}_{i=1}^{n}$ by minimizing a differentiable loss function $L(y, F(x))$, and, is initialized with a constant value, \begin{align*} F_{...
0
votes
0answers
12 views

Aggregation model in stacking with or without initial features

I am implementing a Stacking Ensemble, which works in general, for Supervised problems. Ideally, this was my idea: Train: I take the training-set and after training the N models of the ensemble, I ...
0
votes
0answers
16 views

Adaboost - Is it (really) necessary to plug sample weights into cost function?

I implemented Adaboost using the SAMME algorithm (for multiclass) with Multilayer Perceptron networks as weak learners. For the MLP, i am using ...
2
votes
1answer
209 views

How does bagging reduce variance?

I read this answer. Was still unable to understand how bagging reduces variance. Is there any other way to explain it mathematically to a newbie ? Edit Can anybody explain me this excerpt from the ...
0
votes
0answers
39 views

What is this symbol in Adaboost algorithm (SAMME)?

According to the original paper of SAMME algorithm (Adaboost for multiclass problems), it is described as follows: The general idea is very straightfoward, except for this symbol: The author didn't ...
0
votes
1answer
523 views

Ensembles for Time Series Forecasting with Python

I am building time series models in Python using the statsmodels library. Are there Python resources I can use to build ensembles of such models? (My Google searches suggest that the answer is no.)
0
votes
0answers
14 views

Is there an error in this paper loss?

I'm reading Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifiers and in section 3.1 they describe their entropy margin loss. The goal of this loss is to make the ...
1
vote
3answers
45 views

If 2 expert pieces of advice are independent then both being wrong is lower than individual advice being wrong but isn't both being right also lower?

I was watching this Coursera video on independence which ended up mentioning ensemble. I am trying to wrap my head around probability. If we ask 2 experts whether or not a startup would fail, and we ...
0
votes
0answers
23 views

Ensemble Classifiers modelling different aspects of same instance

Most of the material out there regarding Ensemble of classifiers techniques assumes that each of the classifiers that we want to combine, they are modelling the same aspect of the data. That is, they ...
2
votes
1answer
55 views

Model stacking, Super Learner Algorithm

I've recently started studying ensembles in ML, particularly Super Learner Algorithm. To be honest, although I have read several articles related to this topic, I am a little bit confused. I want to ...

1
2 3 4 5
8