The tag has no usage guidance.

learn more… | top users | synonyms

0
votes
0answers
8 views

How to decide about the number of looks (window size for ensemble averaging) in SAR images?

This question has frustrated me for a while. In order to find an answer I sent an email to prof. Yamaguchi, the author of the paper Four-Component Scattering Power Decomposition With Rotation of ...
1
vote
0answers
32 views
+50

simple statistics of aggregated posterior data after ensemble data assimilation

I have $N$ "4-dimensional" arrays $(x,y,t,c)_{i=1:N}$ containing greenhouse gas emission data, where $(x,y)$ are the spatial coordinates $t$ is the time coordinate (discrete time points) $c$ is the ...
1
vote
1answer
15 views

Combine multiple predictions of binary outcome

I am moving from a single-model prediction of a binary outcome to an aggregate of a small number of models, for example: ...
2
votes
0answers
40 views

Combining multiple classifiers

I am trying to do a binary classification of text articles into {relevant, non-relevant}. The text articles have following features: [[article text, ...
2
votes
0answers
39 views

How to combine regression models?

Say I have three data sets of size $n$ each: $y_1$ = heights of people from the US only $y_2$ = heights of men from the whole world $y_3$ = heights of women from the whole world And I build a ...
0
votes
0answers
28 views

Ensembling with VotingClassifier

I am using VotingClassifier from sklearn.ensemble however i am puzzled with the results. Consider following algorithms: ...
1
vote
0answers
27 views

Reduction of accuracy - Bagging

Bagging is an ensemble method which uses a parallel set of classifiers and then gives consensus output. Usually bagging improves the accuracy of a classification. But if there are situations in which ...
3
votes
1answer
27 views

What happens when Bagging does not have a majority vote?

I have a question regarding the bagging technique used in ensemble learning. Let's assume I have 6 classifiers which could classify a response variable which has 3 finite categories(...
6
votes
1answer
64 views

Gradient Boosting for Linear Regression - why does it not work?

While learning about Gradient Boosting, I haven't heard about any constraints regarding the properties of a "weak classifier" that the method uses to build and ensemble model. However, I could not ...
1
vote
0answers
21 views

How to address Boosting and Bagging decreasing the classification accuracy

For my classification, I use several algorithms available in WEKA, but with limited number of features. I got some accuracy levels with the algorithms I used and I tried improving the accuracies using ...
1
vote
1answer
41 views

Accuracy reduced with Adaboost

I tried using AdaBoost for my classification which is for emotion classification. Without boosting, Random Forest algorithm gave me 42.41% of accuracy. But when I applied AdaBoost along with Random ...
5
votes
1answer
53 views

Boosting neural networks

Well recently I was working on learning boosting algorithms, such as adaboost, gradient boost, and I have known the fact that the most common used weak-learner is trees. I really want to know are ...
1
vote
0answers
38 views

Ensemble Learning: Why is Model Stacking Effective?

Recently, I've become interested in model stacking as a form of ensemble learning. In particular, I've experimented a bit with some toy datasets for regression problems. I've basically implemented ...
1
vote
0answers
15 views

Treating Categorical Variables as Continuous for Random Forest / Adaboost

What's the correct way to deal with categorical variables in packages like sklearn's RF and xgboost? Is there any cons of treating the variables are continuous? E.g. encode class A as 1, class B as ...
3
votes
2answers
61 views

Ensemble classifier methods: should we use the class probabilties or the classification itself in stacking models?

I start to work with ensembe methods these days focusing on stacking. I am wondering whether to us each models class probability ( real number in $[0,1]$) or the classifcation itself (in the binary ...
0
votes
0answers
16 views

Select the baseline methods for ensemble time-series model

I have been reading about ensemble time series models which have often been shown to perform better than single methods. I did not manage to find any reference on how to select the baseline methods. ...
1
vote
0answers
10 views

SVM ensemble with logistic regression

It is possible to average several logistic models in a ensemble using the estimated probabilities of the models. Does it make sense to calculate an ensemble based on the raw SVM score, i.e. the ...
0
votes
0answers
24 views

Generating candidates for Ensemble in machine learning

I have to compare two ensemble methods. First step for me is to create plethora of candidates using different ML algorithms. For this I took small data-sets (with 500-2000 observations) and then ...
0
votes
0answers
13 views

Does it make sense to use a logistic regression model to ensemble probability models

Let $p_1$, $p_2$, ..., $p_k$ denote probabilities for an event with outcome 0 or 1 generated by $K$ distinct models. The linear ensemble of these probabilities is the problem of finding $\alpha_1$, ...
0
votes
0answers
39 views

Ensemble LDA on different feature spaces?

I'm working on a classification problem where I'd like to do the following: I have a space of features that live in $R^m$, and another set of features that are related that live in $R^n$. I want to ...
0
votes
0answers
7 views

Combine textual features with non-textual

Suppose we have spam classification problem and we have some kind of bag of word counts for SVC. What is the proper way to add additional heuristic features, such as length of message\existence of ...
3
votes
1answer
54 views

Random forest regression - residuals correlated with response

I am trying to use Random Forest regression. I have a response variable: y = rnorm(10000, mean=0, sd=3) And a few predictor variables (which are just the ...
1
vote
1answer
71 views

RandomForest returns wrong number of trees

I am trying to predict churn of customer using Random Forest. I am following this article. However I am stuck with part 8 (ln[8] and Out[8]) - I want to use only 10 estimators (to have only 10 trees) ...
0
votes
1answer
158 views

Ensemble models in R

I have a clinical dataset (1400 cases) and I applied 4 data mining techniques (ANN, Decision Tree, SVM, Logistic Regression) to predict the binary outcome (Yes, No). Now, I want to improve prediction ...
1
vote
0answers
37 views

How to eliminate noise variables when using ensemble prediction methods like randomGLM in R?

The task involves predicting a binary outcome in a small data set (sample sizes of 20-70) using many (>100) variables as potential predictors. The main problem is that the number of predictors is much ...
1
vote
0answers
45 views

Stacking ensembles to improve prediction

I recently read this blog and it has many ideas for ensembling various models. I created three models for my training data, random forest model, SVM model and a KNN model. However when I use linear ...
1
vote
1answer
78 views

How does one interpret the predictions of the Super Learner algorithm in R when the outcome is binary?

Implementing the binary outcome (0 or 1) toy example from the Super Learner documentation produces a vector of values between 0 and 1 for the SL.predict object. My ...
3
votes
1answer
44 views

Uniform implementation of Random Forest and Adaboost

If I have a set RF of Decision Trees trained using a Random Forest algorithm and a set AB of Decision Trees (Stumps), ...
2
votes
1answer
53 views

Combining different classifiers yields lower accuracy than a Random Forest alone

I used the following classifiers (with accuracies): Random Forest - 85 % SVM - 78 % Adaboost - 82% Logistic regression - 80% When I used voting from the above classifiers for final ...
0
votes
0answers
36 views

Ensemble models: Logistic and Linear Regression

I'm looking to predict customer churn: reduction in revenue in one time period vs the same time period last year. I've built a logistic regression model to classify customers as churners and ...
2
votes
1answer
80 views

Are there any new approaches to generating random examples in Combined Multiple Models (CMMs)?

A recent question got me wondering about the current state of the art in Combined Multiple Models (CMMs). I am familiar with work by Domingos in the 90s, such as this ICML paper: Domingos, Pedro. ...
0
votes
0answers
48 views

Avoid overfitting in an ensemble classification model

We are building a classifier on a binary outcome. The priors are a roughly 17/83 split in a sample of 3000 observations. There are about 500 predictors, a mix of categorical and integer. We split ...
0
votes
0answers
22 views

Chaining ensemble networks

I'm working on prediction construction costs and have encountered an interesting question... At present, I'm using a data set that describes the construction work by the job location, time of year, ...
2
votes
1answer
84 views

Sampling : Gradient Boosting Tree

I have a question regarding the algorithm of Gradient Boosting Tree. I understand Simple tree is built for only a randomly selected sub sample of the full data set (random without replacement). Each ...
1
vote
1answer
28 views

How to determine if the errors made by the classifiers are uncorrelated

I am working on ensemble methods to improve the Area under the ROC curve in an experiment. In Ensemble Methods in Machine Learning ", Dietterich says " A necessary and suficient condition for an ...
0
votes
1answer
34 views

Combining multiple OLS Regressions

I have a single output $y$, and multiple inputs $x_1, x_2,\dots,x_n$. I am running online(streaming) regression, which would be complicated with many inputs. So, to go around it, I want to have $n$ ...
1
vote
1answer
29 views

How can I generate ensembles from spatially correlated PDFs?

So I have a grid where every grid point has PDF of a variable (precipitation). The PDFs are spatially correlated. What is the best way for generating ensembles that are spatially and temporally ...
0
votes
1answer
64 views

whats a difference between multiple kernel learning and ensemble learning?

From wiki: Ensemble methods use multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms Multiple kernel learning ...
2
votes
1answer
70 views

Ensembling Logistic Regressions Fit on Different Datasets

I would like to predict a binary response variable $Y_i$ using sets of predictors $\textbf{X}_{1i}, \textbf{X}_{2i}, \textbf{X}_{3i}$ for $i=1,\dots,n$. Each $\textbf{X}$ contains a few dozen ...
0
votes
1answer
144 views

Using bagged ensemble of regression trees, feature selection based on feature importance

I am working on relating aesthetic scores of given images (about 17k training+validation samples and 280 image features) and getting best result using ensemble of CARTs. Beside achieveing a good ...
1
vote
1answer
54 views

Adaboost for numeric dataset

I have been trying to fit Adaboost to work with continuous valued data set and the more I read the more I keep getting confused. I have read about the multiclass Adaboost with log(K-1) addition to ...
0
votes
2answers
50 views

Name some techniques similar to Random Forests

I'm interested in what techniques are out there that are similar to, but not the same as, Random Forests. Either for classification or regression or both. Particularly interested in techniques which ...
2
votes
1answer
27 views

Simple voting scheme using confidence for each vote

I am doing classification by splitting each observation into 14 subparts and then classifying each of these subparts individually. The overall classification of the observation is then performed using ...
14
votes
3answers
271 views

When should I not use an ensemble classifier?

In general, in a classification problem where the goal is to accurately predict out-of-sample class membership, when should I not to use an ensemble classifier? This question is closely related to ...
2
votes
0answers
80 views

Limitations of ensemble selection from libraries

Question related to the approach in Caruana's paper: "Ensemble Selection from Libraries of Models" (linked below) http://www.cs.cornell.edu/~caruana/ctp/ct.papers/caruana.icml04.icdm06long.pdf Seems ...
3
votes
3answers
288 views

How to combine weak classfiers to get a strong one?

Let as assume that we have a binary classification problem. We also have several classifiers. Instead of assigning a vector to a class (0 or 1) each classifier returns a probability that a given ...
1
vote
0answers
138 views

Boosting Explained

I'm a newbie trying to learn Boosting. The examples I found online are quite confusing. Is there a simple tutorial somewhere that explains ...
1
vote
1answer
76 views

Skewed Classification Problem

So I've read around and seen this is a problem. I have a classification problem and 12 variables ... I'm working on getting more, but even if l get the number to 20-30 I feel like the problem will ...
1
vote
1answer
1k views

Buiding Ensemble model

I'm new to ensemble model. Suppose I've KNN models like this - (in R) library(class) ...
0
votes
3answers
122 views

Comparing SVM Model

Pardon my understanding of SVMs as it is very little. We often hear of ensemble classifiers and stuff like this. Say if i were to have 3 different SVM Models for the same dataset predicting a ...