The tag has no wiki summary.

learn more… | top users | synonyms

4
votes
0answers
22 views

Polling vs averaging in Random Forest models

Why is it that for Random Forest we take the average vote from each classifier in the ensemble rather than the average probability from each classifier in the ensemble? Is there theory behind why ...
1
vote
0answers
12 views

Proper way to stack models when some models aren't always applicable?

Suppose you have two (or more) models that you want to ensemble together. However, some of the models are trained specifically on very specialized subsets of your data. If you do the stacking with a ...
0
votes
0answers
19 views

Stacked Generalization Ensemble Algorithm for regression

I am using stacked generalization(Rupert 1992) for combining multiple(8) heterogeneous base learners for regression. What I understand from the pseudo codes that Train the 8 learners on 8 instances ...
1
vote
0answers
18 views

Ensemble model performs better with worse performing consitutent models?

I have a forecast model I am developing that uses some very unreliable input data, missing data (due to sensors or comms failures) is the rule, not an exception. The quantity being forecast is a daily ...
2
votes
1answer
150 views

Why not always use ensemble learning?

It seems to me that ensemble learning WILL always give better predictive performance than with just a single learning hypothesis. So, why don't we use them all the time? My guess is because of ...
0
votes
1answer
40 views

What is this ensemble learning technique called?

For example, I trained two models: one with SVM and one with KNN. Final Prediction = 0.4*KNN + 0.6*SVM Is this considered blending?
0
votes
1answer
79 views

What is the equation for random forest?

I need an equation for random forest so that I can score fresh data I receive every week, based on beta estimates I got after building model using this ensemble methodology. Every week I do not want ...
0
votes
1answer
27 views

What method to use for cluster identification ?

This question is from a confused novice. I have a data set with where each point is located in a 2-D space defined by two objectives (say, X and Y). I wish to identify a set of points from this space ...
4
votes
2answers
172 views

Ensembles of Ensembles?

I like the idea of ensemble learners, especially Bagging, but I always wondered as why they are not the most powerful learners since they have a clean motivation. I don't have the answer to that ...
0
votes
0answers
14 views

combining classifiers trained on different data

Could you please recommend methods for combining classifiers, which were trained on different patients? For example, there are 10 patients and for each of them I trained a binary classifier. Now I ...
2
votes
1answer
96 views

k-fold Cross validation of ensemble learning

I am confused about how to partition the data for k-fold cross validation of ensemble learning. Assuming I have an ensemble learning framework for classification. My first layer contains the ...
1
vote
2answers
86 views

Combine decision trees from GBM to reduce output

I am curious if any research has been conducted to efficiently combine trees resulting from a gradient boosting process. I routinely run a process that generates 20 or 30 thousand trees in R. I then ...
1
vote
0answers
41 views

How to combine predictions in ensemble

I am trying to learn more about how to build ensembles of predictions in R and coming to a roadblock, and am hoping one can offer guidance. I often read about people automatically identifying how ...
1
vote
0answers
70 views

What are the latest methods to generate ensembles?

I am working with ensembles, and I'm willing to go deep inside the work. I have historical records of: Observations of one variable Historical forecasts for the same variable For future ...
3
votes
2answers
66 views

Combining models for prediction based on residual performance

I have never read or seen someone do this before, so I wanted to pose the question here. Suppose I fit a basic linear model, $\text{price of house} = \beta_0 + \beta_1*\text{taxes} + ...
0
votes
2answers
65 views

Ensemble of models with different feature spaces

BACKGROUND I have data in which the dependent variable is binary with a highly-skewed distribution: <1% records are 1 (doers), >99% records are 0 (non-doers). I'm using logistic regression to ...
0
votes
0answers
26 views

Building Ensemble Models Practical Question

I understand the general idea of stacking several models and combining them to create a model that might perform better than any of the individual models that it is composed of. My question is ...
1
vote
0answers
36 views

Ensemble learning combined with item response theory

I am recently learning item response theory, which is always used to select some combinations of tests for students of different specific levels. Then I have an immature idea. Could I apply item ...
2
votes
0answers
59 views

Dissimilarity with earlier features part of cost function

I am using a RandomForest on features (pixels) of images, and I am considering adding cost for "similarity to already other included features" to the cost function. Imagine you have a current RF ...
0
votes
0answers
29 views

Early split decision criteria for fast random (regression) forest estimation

Suppose I am on a node in a $regression$ tree and I am using running estimates of $\sum_{i \in Region_1} (y_i - mean(y_i)_{Region1})^2$ (and the same for Region 2) to determine whether to split the ...
11
votes
3answers
824 views

Can Random Forest Methodology be Applied to Linear Regressions?

Random Forests work by creating an ensemble of decision trees where each tree is created using a bootstrap sample of the original training data (sample of both input variables and observations). Can ...
3
votes
2answers
156 views

Why are Random Forests splitted based on m random features?

I was watching the following tutorial on Random forests and it says that, "at each node, choose random subset of m features and only consider splitting on those features" I do not understand why we ...
0
votes
0answers
34 views

Comparing estimates from different models

How would I compare (posterior) probability estimates of a document from n classifiers trained separately on different datasets (the # of classes in each classifier is different). Is there a way to ...
1
vote
0answers
77 views

Proper splitting of data set for Ensemble methods

I have 10,000 documents. Each document has a label ($Y$) that is either $0$ or $1$ (the 0-1 split is pretty much 50/50 over my 10,000 documents). Each document has 10 fields. Each field can have any ...
5
votes
1answer
106 views

Do ensemble techniques increase VC-dimension?

Techniques like Adaboost use a ensemble of weak classifiers to obtain a "better" classifier. Does(Can) the final classifier have a greater VC-dimension than the weak classifier? An intuitive ...
1
vote
0answers
77 views

Gradient boosting algorithm (steps) question

So, far I have read following regarding boosting: Boosting is an ensemble technique. Train learner sequentially, where early learners fit simple models to the data. Analyze data for errors, that ...
1
vote
1answer
255 views

Matrix Factorization Model for recommender systems how to determine number of latent features?

I am trying to design a matrix factorization technique for a simple user-item, rating recommender system. I have 2 questions about this. First in a simple implementation that I saw of matrix ...
0
votes
0answers
35 views

What sort of learning algorithm to use?

I am looking to perform a simple task, Given restaurant_reviews from week 1 to week n for a particular restaurant,I look to perform prediction about the number of reviews the restaurant will get ...
0
votes
0answers
25 views

Negative coefficient in model stacking

I've some data I'm modeling with a couple of different approaches. Random Forests on one side, a logistic regression on the other side. I wanted to create an ensemble model using the predicted ...
0
votes
1answer
71 views

Using a logistic model on the estimates of several other classification models

I'm working on a classification model that will predict whether a sales opportunity will end up 'won' or 'lost', given various attributes of the opportunity. I've been using my training data to build ...
2
votes
1answer
124 views

Is there a well-defined class of ensemble methods?

Ensemble methodology's main aim is to somehow aggregate or summarize estimates from multiple models. In some cases this is aggregating different bootstrap estimates or Monte Carlo estimates, but ...
6
votes
0answers
541 views

Using LASSO on random forest

I would like to create a random forest using the following process: Build a tree on a random samples of the data and features using information gain to determine splits Terminate a leaf node if it ...
4
votes
2answers
315 views

On combining SVMs

Suppose we have a supervised training set $T=\{ (x_1, y_1),\dots, (x_n,y_n)\}$ where $x_i$ is an example and $y_i \in \{-1,+1\}$ is its label. Further suppose that examples are only observable through ...
1
vote
0answers
56 views

One-against-all probability values into a multiple class probability value?

I have a 10-class classification problem. I've approached the problem as a set of one-against-all binary problems. For each class I've built a MLP neural network that provides a probability estimate ...
4
votes
0answers
110 views

Limits to tree-based ensemble methods in small n, large p problems?

Long time grazer, first time poster. I'm hoping to gather people's opinion on the following: Tree-based ensemble methods such as Random Forest, and subsequent derivatives (e.g., conditional forest), ...
6
votes
3answers
493 views

Ensemble time series model

I need to automate time-series forecasting, and I don't know in advance the features of those series (seasonality, trend, noise, etc). My aim is not to get the best possible model for each series, ...
0
votes
1answer
59 views

Defintion for model diversity?

Two models are diverse if they make prediction errors on different instances. I know there are different measures to quantify diversity, however, I'm looking for formal conceptual definition of what ...
1
vote
1answer
719 views

Looking for examples or alternatives to R RuleFit ensemble package

Does anyone know of any good example code illustrations for the rulefit Rule Based Learning Ensembles package? The documentation is incredibly lacking. I was guided to the package by this paper. If ...
2
votes
1answer
347 views

Which are the most effective clustering ensembles?

In supervised learning, there are some ensemble methods that overcome others significantly (adaboost or random forests to mention some). Few years later, also ensembles in unsupervised learning were ...
3
votes
1answer
178 views

How to determine the best number of weak classifiers to use in adaboost without overfitting the data

I was thinking by using validation but not quite sure how to go with it. Please list some papers or ideas on how. This is for multiclass problem (using one vs all approach). I think each ...
1
vote
1answer
462 views

How to convert multiple ranking scores into a probability distribution?

I would like to create a topic distribution for a document. The current model I am trying to implement is: for each sentence in the document, I am getting a topic assignment with a score, e.g. "1st ...
2
votes
1answer
339 views

Concept of iterations in Adaboost

I can't seem to get my head around "iterations" in Adaboost. Are they analogous to weak classifiers that are used for Boosting? I've seen many examples of Adaboost where a programmers use a Single ...
0
votes
1answer
133 views

Accuracy of classifiers with Adaboost

Does Adaboost ensure that resultant accuracy is more than or at least equal to current accuracies? What happens if Classifier A performs badly and the weights are accordingly updated and the next ...
8
votes
4answers
2k views

Resources for learning how to implement ensemble methods

I understand theoretically (sort of) how they would work, but am not sure how to go about actually making use an ensemble method (such as voting, weighted mixtures, etc.). What are good resources ...
2
votes
1answer
128 views

Weighting variables for an index

I have been tasked with trying to modify our current "index" which basically takes 4 observations per person and calculates a score based on what they achieve. Here is how the score is created (all ...
7
votes
3answers
3k views

Stacking/ensembling models with caret

I often find myself training several different predictive models using caret in R. I'll train them all on the same cross validation folds, using ...
2
votes
1answer
335 views

Ensembling regression models

I'm working on a securities pricing project and have a bunch of models I'd like to stack/ensemble together. I've been using simple linear regression in R (the lm() ...
1
vote
0answers
75 views

Tractability of mutual information-augmented ensemble classification algorithms

I am seeking to augment random forest classification using Shannon-Weaver mutual information as a metaheuristic to partition candidate datasets. Specifically, I am trying to determine if such an ...
2
votes
1answer
163 views

What are the strongest boosting alternatives to Adaboost?

Whenever boosting is brought up, Adaboost is the first algorithm to be listed. What are the most popular boosting algorithms that aren't Adaboost?
6
votes
2answers
384 views

How are classifications merged in an ensemble classifier?

How does an ensemble classifier merge the predictions of its constituent classifiers? I'm having difficulty finding a clear description. In some code examples I've found, the ensemble just averages ...