The tag has no usage guidance.

learn more… | top users | synonyms

0
votes
0answers
7 views

How can one quantify the variable importance dilution effect in random forests (and similar statistical learning methods)?

In Applied Predictive Modelling (Kuhn, Johnson, 2013, p 202), the authors refer to a dilution effect whereby compared to a single tree or a classical regression technique, the difference in importance ...
1
vote
1answer
26 views

Why are gradient boosting regression trees good candidates for ranking problems?

I have been reading up on gradient boosting machines, and in particular GBRT's. I've come across numerous mentions (and finally tracked down some papers) on applying these models to ranking problems - ...
0
votes
0answers
5 views

kNN used as a metafeature for future ensemble

Lately I've saw a lot of classification approaches for large datasets that involved ensemble methods, most of them using kNN. If I don't miss-understand the algorithm, it is meant to use a portion of ...
2
votes
1answer
34 views

Ensemble models perform worse than single one

In my model testing, I tried to use model ensembling (blending in this case) to get better results. However the ensemble cannot beat single RandomForrestClassifier. In first layer, I train ...
0
votes
0answers
6 views

Can I avoid overfit in an ensemble model using a bootstrap with small samples as training sets?

Let´s suppose that t I have a dataset with 250 data points and I want to train an ensemble. If I choose to train each classifier of the ensemble with a small bootstrap sample (10) of the original ...
1
vote
1answer
34 views

How to proceed with building an ensemble classifier using Naive Bayes, TAN and Logistic Regression in R

I'm relatively new to machine learning (started about 5 months ago), and I'm looking at potentially implementing an ensemble classifier as part of my research. I have built 3 models that I use to ...
1
vote
2answers
34 views

ensemble model of uni-variate linear regression models

Calculating univariate linear regression and correlation is simple in SQL. It can be done like so: ...
2
votes
0answers
16 views

Multiclass classification one versus one with ensemble

If I use an ensemble, which consists of four classifiers, in order to classify my data into three classes. Further, suppose I use the one versus one strategy. My question is: How to fuse the ...
0
votes
0answers
8 views

Combining classification results

Let's say I have a binary classification problem and several models $M_1, \dots, M_n$ to predict the classification results. In my setting the outputs of these models are quantitative (some of them ...
1
vote
0answers
25 views

ensemble methods:voting with average of probabilities in weka

output attribute is risky patient. Values are yes and no.if yes then patient is risky and if no then patient is not risky. If I am combining 3 classifier for classification model in weka, and if ...
0
votes
1answer
15 views

Weighing classifiers based on the cross validation accuracy

Suppose there exist n individual classifiers which have different parameters and have been trained on the same data. In order to build an ensemble of these classifiers, is it an optimal method to ...
0
votes
0answers
32 views

How to evaluate stacking ensemble model vs. other models with 10-fold cross-validation?

I've been comparing various predictive models for both continuous and binary outcomes for a health care model. So far 10-fold cross-validation has been useful: training models on 9/10 of the analysis ...
1
vote
0answers
24 views

Model selection in ensembles

I'm trying to build an ensemble for a ML problem where fast prediction time is critical. So I'm interested in keeping my set of level-0 models for the ensemble pruned. Which measures can I use to ...
1
vote
0answers
22 views

Boosting in unsupervised learning - methods and use cases

I'm looking for methods and uses cases for applying boosting or other ensemble methods for unsupervised learning Examples of such methods and use cases are: Boosting density estimation Saharon ...
3
votes
0answers
13 views

Theory of correlation and weighing when ensabling models

I'm ensabling models together to improve the overall performance. At the moment, I'm weighing each model by its performance under cross-validation, and this works reasonably well. Clearly the best ...
0
votes
0answers
13 views

Classification: estimate how many people are in a household through account transactions

I want to estimate how many people are in a household by looking at account transactions. It would be also interesting to understand if in the household there are children. I thought that a possible ...
3
votes
1answer
68 views

stacking and blending of regression models

I am self-studying blending and stacking, and am especially interested in this in the context of regression models. I have been reading a number of the stacking, blending and bagging links posted on ...
0
votes
0answers
25 views

Combining strong and weak learners

Our NGO is struggling with the following problem: For an ecological application, we have tried to program a deep neural net to estimate the size and weight of images of birds that we have captured on ...
0
votes
0answers
11 views

How to decide about the number of looks (window size for ensemble averaging) in SAR images?

This question has frustrated me for a while. In order to find an answer I sent an email to prof. Yamaguchi, the author of the paper Four-Component Scattering Power Decomposition With Rotation of ...
1
vote
0answers
40 views

simple statistics of aggregated posterior data after ensemble data assimilation

I have $N$ "4-dimensional" arrays $(x,y,t,c)_{i=1:N}$ containing greenhouse gas emission data, where $(x,y)$ are the spatial coordinates $t$ is the time coordinate (discrete time points) $c$ is the ...
1
vote
1answer
24 views

Combine multiple predictions of binary outcome

I am moving from a single-model prediction of a binary outcome to an aggregate of a small number of models, for example: ...
2
votes
0answers
70 views

Combining multiple classifiers

I am trying to do a binary classification of text articles into {relevant, non-relevant}. The text articles have following features: [[article text, ...
3
votes
0answers
59 views

How to combine regression models?

Say I have three data sets of size $n$ each: $y_1$ = heights of people from the US only $y_2$ = heights of men from the whole world $y_3$ = heights of women from the whole world And I build a ...
0
votes
0answers
43 views

Ensembling with VotingClassifier

I am using VotingClassifier from sklearn.ensemble however i am puzzled with the results. Consider following algorithms: ...
1
vote
0answers
28 views

Reduction of accuracy - Bagging

Bagging is an ensemble method which uses a parallel set of classifiers and then gives consensus output. Usually bagging improves the accuracy of a classification. But if there are situations in which ...
3
votes
1answer
42 views

What happens when Bagging does not have a majority vote?

I have a question regarding the bagging technique used in ensemble learning. Let's assume I have 6 classifiers which could classify a response variable which has 3 finite categories(...
6
votes
1answer
115 views

Gradient Boosting for Linear Regression - why does it not work?

While learning about Gradient Boosting, I haven't heard about any constraints regarding the properties of a "weak classifier" that the method uses to build and ensemble model. However, I could not ...
1
vote
0answers
52 views

How to address Boosting and Bagging decreasing the classification accuracy

For my classification, I use several algorithms available in WEKA, but with limited number of features. I got some accuracy levels with the algorithms I used and I tried improving the accuracies using ...
1
vote
1answer
60 views

Accuracy reduced with Adaboost

I tried using AdaBoost for my classification which is for emotion classification. Without boosting, Random Forest algorithm gave me 42.41% of accuracy. But when I applied AdaBoost along with Random ...
6
votes
1answer
87 views

Boosting neural networks

Well recently I was working on learning boosting algorithms, such as adaboost, gradient boost, and I have known the fact that the most common used weak-learner is trees. I really want to know are ...
1
vote
0answers
63 views

Ensemble Learning: Why is Model Stacking Effective?

Recently, I've become interested in model stacking as a form of ensemble learning. In particular, I've experimented a bit with some toy datasets for regression problems. I've basically implemented ...
1
vote
0answers
25 views

Treating Categorical Variables as Continuous for Random Forest / Adaboost

What's the correct way to deal with categorical variables in packages like sklearn's RF and xgboost? Is there any cons of treating the variables are continuous? E.g. encode class A as 1, class B as ...
4
votes
2answers
90 views

Ensemble classifier methods: should we use the class probabilties or the classification itself in stacking models?

I start to work with ensembe methods these days focusing on stacking. I am wondering whether to us each models class probability ( real number in $[0,1]$) or the classifcation itself (in the binary ...
0
votes
0answers
21 views

Select the baseline methods for ensemble time-series model

I have been reading about ensemble time series models which have often been shown to perform better than single methods. I did not manage to find any reference on how to select the baseline methods. ...
1
vote
0answers
13 views

SVM ensemble with logistic regression

It is possible to average several logistic models in a ensemble using the estimated probabilities of the models. Does it make sense to calculate an ensemble based on the raw SVM score, i.e. the ...
0
votes
0answers
27 views

Generating candidates for Ensemble in machine learning

I have to compare two ensemble methods. First step for me is to create plethora of candidates using different ML algorithms. For this I took small data-sets (with 500-2000 observations) and then ...
0
votes
0answers
17 views

Does it make sense to use a logistic regression model to ensemble probability models

Let $p_1$, $p_2$, ..., $p_k$ denote probabilities for an event with outcome 0 or 1 generated by $K$ distinct models. The linear ensemble of these probabilities is the problem of finding $\alpha_1$, ...
0
votes
0answers
47 views

Ensemble LDA on different feature spaces?

I'm working on a classification problem where I'd like to do the following: I have a space of features that live in $R^m$, and another set of features that are related that live in $R^n$. I want to ...
0
votes
0answers
7 views

Combine textual features with non-textual

Suppose we have spam classification problem and we have some kind of bag of word counts for SVC. What is the proper way to add additional heuristic features, such as length of message\existence of ...
3
votes
1answer
74 views

Random forest regression - residuals correlated with response

I am trying to use Random Forest regression. I have a response variable: y = rnorm(10000, mean=0, sd=3) And a few predictor variables (which are just the ...
1
vote
1answer
85 views

RandomForest returns wrong number of trees

I am trying to predict churn of customer using Random Forest. I am following this article. However I am stuck with part 8 (ln[8] and Out[8]) - I want to use only 10 estimators (to have only 10 trees) ...
0
votes
1answer
245 views

Ensemble models in R

I have a clinical dataset (1400 cases) and I applied 4 data mining techniques (ANN, Decision Tree, SVM, Logistic Regression) to predict the binary outcome (Yes, No). Now, I want to improve prediction ...
1
vote
0answers
40 views

How to eliminate noise variables when using ensemble prediction methods like randomGLM in R?

The task involves predicting a binary outcome in a small data set (sample sizes of 20-70) using many (>100) variables as potential predictors. The main problem is that the number of predictors is much ...
1
vote
0answers
73 views

Stacking ensembles to improve prediction

I recently read this blog and it has many ideas for ensembling various models. I created three models for my training data, random forest model, SVM model and a KNN model. However when I use linear ...
1
vote
1answer
145 views

How does one interpret the predictions of the Super Learner algorithm in R when the outcome is binary?

Implementing the binary outcome (0 or 1) toy example from the Super Learner documentation produces a vector of values between 0 and 1 for the SL.predict object. My ...
3
votes
1answer
53 views

Uniform implementation of Random Forest and Adaboost

If I have a set RF of Decision Trees trained using a Random Forest algorithm and a set AB of Decision Trees (Stumps), ...
2
votes
1answer
89 views

Combining different classifiers yields lower accuracy than a Random Forest alone

I used the following classifiers (with accuracies): Random Forest - 85 % SVM - 78 % Adaboost - 82% Logistic regression - 80% When I used voting from the above classifiers for final ...
0
votes
0answers
50 views

Ensemble models: Logistic and Linear Regression

I'm looking to predict customer churn: reduction in revenue in one time period vs the same time period last year. I've built a logistic regression model to classify customers as churners and ...
3
votes
1answer
87 views

Are there any new approaches to generating random examples in Combined Multiple Models (CMMs)?

A recent question got me wondering about the current state of the art in Combined Multiple Models (CMMs). I am familiar with work by Domingos in the 90s, such as this ICML paper: Domingos, Pedro. ...
0
votes
0answers
76 views

Avoid overfitting in an ensemble classification model

We are building a classifier on a binary outcome. The priors are a roughly 17/83 split in a sample of 3000 observations. There are about 500 predictors, a mix of categorical and integer. We split ...