The tag has no usage guidance.

learn more… | top users | synonyms

2
votes
1answer
31 views

Sampling : Gradient Boosting Tree

I have a question regarding the algorithm of Gradient Boosting Tree. I understand Simple tree is built for only a randomly selected sub sample of the full data set (random without replacement). Each ...
1
vote
1answer
13 views

How to determine if the errors made by the classifiers are uncorrelated

I am working on ensemble methods to improve the Area under the ROC curve in an experiment. In Ensemble Methods in Machine Learning ", Dietterich says " A necessary and suficient condition for an ...
0
votes
1answer
25 views

Combining multiple OLS Regressions

I have a single output $y$, and multiple inputs $x_1, x_2,\dots,x_n$. I am running online(streaming) regression, which would be complicated with many inputs. So, to go around it, I want to have $n$ ...
1
vote
1answer
25 views

How can I generate ensembles from spatially correlated PDFs?

So I have a grid where every grid point has PDF of a variable (precipitation). The PDFs are spatially correlated. What is the best way for generating ensembles that are spatially and temporally ...
0
votes
1answer
29 views

whats a difference between multiple kernel learning and ensemble learning?

From wiki: Ensemble methods use multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms Multiple kernel learning ...
2
votes
1answer
57 views

Ensembling Logistic Regressions Fit on Different Datasets

I would like to predict a binary response variable $Y_i$ using sets of predictors $\textbf{X}_{1i}, \textbf{X}_{2i}, \textbf{X}_{3i}$ for $i=1,\dots,n$. Each $\textbf{X}$ contains a few dozen ...
0
votes
1answer
57 views

Using bagged ensemble of regression trees, feature selection based on feature importance

I am working on relating aesthetic scores of given images (about 17k training+validation samples and 280 image features) and getting best result using ensemble of CARTs. Beside achieveing a good ...
1
vote
1answer
42 views

Adaboost for numeric dataset

I have been trying to fit Adaboost to work with continuous valued data set and the more I read the more I keep getting confused. I have read about the multiclass Adaboost with log(K-1) addition to ...
0
votes
2answers
35 views

Name some techniques similar to Random Forests

I'm interested in what techniques are out there that are similar to, but not the same as, Random Forests. Either for classification or regression or both. Particularly interested in techniques which ...
2
votes
1answer
18 views

Simple voting scheme using confidence for each vote

I am doing classification by splitting each observation into 14 subparts and then classifying each of these subparts individually. The overall classification of the observation is then performed using ...
11
votes
3answers
226 views

When should I not use an ensemble classifier?

In general, in a classification problem where the goal is to accurately predict out-of-sample class membership, when should I not to use an ensemble classifier? This question is closely related to ...
2
votes
0answers
56 views

Limitations of ensemble selection from libraries

Question related to the approach in Caruana's paper: "Ensemble Selection from Libraries of Models" (linked below) http://www.cs.cornell.edu/~caruana/ctp/ct.papers/caruana.icml04.icdm06long.pdf Seems ...
0
votes
0answers
76 views

How to combine weak classfiers to get a strong one?

Let as assume that we have a binary classification problem. We also have several classifiers. Instead of assigning a vector to a class (0 or 1) each classifier returns a probability that a given ...
1
vote
0answers
85 views

Boosting Explained

I'm a newbie trying to learn Boosting. The examples I found online are quite confusing. Is there a simple tutorial somewhere that explains ...
1
vote
1answer
61 views

Skewed Classification Problem

So I've read around and seen this is a problem. I have a classification problem and 12 variables ... I'm working on getting more, but even if l get the number to 20-30 I feel like the problem will ...
1
vote
1answer
431 views

Buiding Ensemble model

I'm new to ensemble model. Suppose I've KNN models like this - (in R) library(class) ...
0
votes
3answers
87 views

Comparing SVM Model

Pardon my understanding of SVMs as it is very little. We often hear of ensemble classifiers and stuff like this. Say if i were to have 3 different SVM Models for the same dataset predicting a ...
2
votes
1answer
83 views

Minimizing Curve fit for predictive model

Let's assume we've found 100 independent variables that can predict y. Each of those independent variable are close to uncorrelated and they are all curve fitted. Using any single one to predict y ...
2
votes
1answer
192 views

Model Stacking algorithm

I'm trying the stacking method to see if it improves my results, but before using some R package, I decided to code it by myself. Here's a pseudocode of what I'm doing: ...
3
votes
3answers
533 views

Ensemble of different kinds of regressors using scikit-learn (or any other python framework)

I am trying to solve the regression task. I found out that 3 models are working nicely for different subsets of data: LassoLARS, SVR and Gradient Tree Boosting. I noticed that when I make predictions ...
0
votes
0answers
144 views

Why do Matlab's TreeBagger and fitensemble with 'bag' and same parameters give different predictions

Matlab's Statistical Toolbox has two bagging tree algorithms implementation: Tree Bagger Fitensemble (see 'Bag' method) I am currently using (1) for a Regression problem. However, I would like to ...
0
votes
0answers
38 views

adaboost with multiple classification algorithms

Up to now I saw that all adaboost implementations use single classification algorithm and a training dataset as input and then creates multiple classification models by re-sampling dataset and uses ...
0
votes
0answers
131 views

Combining multiple feature subsets through ensemble classification methods?

I have a set of $N$ samples to be classifies in a binary classification problem. I have extracted features from these samples from 4 different perspectives (views) of every samples. Hence I have 4 ...
2
votes
1answer
179 views

what is the effect of bootstap resampling in bagging algorithm(ensemble learning)?

In ensemble learning with bagging, why is it important to do bootstrap resampling (sampling with replacement) instead of just sub-sampling (sampling without replacement)?
0
votes
1answer
74 views

Confused about cross validation for model stacking

I'm reading section 8.8 of Elements of Statistical Learning, and though I keep reading the section on calculating the ensemble weights I'm missing something. It says that the stacking weights are ...
1
vote
0answers
39 views

What techniques are used to prevent overfitting in DSN

A Deep Stacked Network (DSN), is a ensemble learner, which roughly works by training a single hidden layer neural network on the inputs and target outputs, then training another which takes an input ...
0
votes
0answers
106 views

credit scoring - fraud scoring

I have been asked to build a credit scoring model and we are relying on several Machine Learning API, in order to build our feature vectors. One of these API is MinFraud. However, as they provide us ...
1
vote
0answers
58 views

What's the potential reason that by combining two feature sets the performance of random forest dropped?

I am building random forests on high dimensional, sparse, and class unbalanced training datasets (around 500 - 5000 examples) using two different feature sets. I did stratified 10-fold cross ...
1
vote
0answers
22 views

Assessing predictor contribution to model output

Many of machine learning methods are considered as "black boxes". Examples of such methods are SVM, Neural Networks, Random forests etc. One may apply sensitivity analysis techniques (as described for ...
1
vote
2answers
103 views

Analysis for checking if an Ensemble model is a better fit for a dataset than Primitive model

I have a dataset and have the option to apply either GLM (primitive) or a Random Forest (ensemble). So far the Random Forest is giving way better results than the GLM. As it is generally believed that ...
0
votes
0answers
18 views

In ensemble predictions, what is reliability diagram?

I have recently heard the term reliability diagram used with regards to the analysis of ensemble predictions. What does it show and how is it calculated?
0
votes
1answer
37 views

What statistical analyses should one perform on ensemble forecasts (given a measurement)?

I have an ensemble of time-series predicting a scalar variable. Additionally, I have a measurement time series of this scalar variable. Which statistical analyses could and/or should I perform to ...
0
votes
1answer
149 views

Suggestions needed about classifier fusion

I'm working on a classification problem which involves two classifier to observe a single event. I'm providing a high level description of the problem without going into the technical details (the ...
4
votes
0answers
50 views

Polling vs averaging in Random Forest models

Why is it that for Random Forest we take the average vote from each classifier in the ensemble rather than the average probability from each classifier in the ensemble? Is there theory behind why ...
1
vote
0answers
24 views

Proper way to stack models when some models aren't always applicable?

Suppose you have two (or more) models that you want to ensemble together. However, some of the models are trained specifically on very specialized subsets of your data. If you do the stacking with a ...
1
vote
0answers
50 views

Ensemble model performs better with worse performing consitutent models?

I have a forecast model I am developing that uses some very unreliable input data, missing data (due to sensors or comms failures) is the rule, not an exception. The quantity being forecast is a daily ...
4
votes
1answer
400 views

Why not always use ensemble learning?

It seems to me that ensemble learning WILL always give better predictive performance than with just a single learning hypothesis. So, why don't we use them all the time? My guess is because of ...
0
votes
1answer
54 views

What is this ensemble learning technique called?

For example, I trained two models: one with SVM and one with KNN. Final Prediction = 0.4*KNN + 0.6*SVM Is this considered blending?
1
vote
1answer
644 views

What is the equation for random forest?

I need an equation for random forest so that I can score fresh data I receive every week, based on beta estimates I got after building model using this ensemble methodology. Every week I do not want ...
0
votes
1answer
29 views

What method to use for cluster identification ?

This question is from a confused novice. I have a data set with where each point is located in a 2-D space defined by two objectives (say, X and Y). I wish to identify a set of points from this space ...
4
votes
2answers
254 views

Ensembles of Ensembles?

I like the idea of ensemble learners, especially Bagging, but I always wondered as why they are not the most powerful learners since they have a clean motivation. I don't have the answer to that ...
4
votes
1answer
998 views

k-fold Cross validation of ensemble learning

I am confused about how to partition the data for k-fold cross validation of ensemble learning. Assuming I have an ensemble learning framework for classification. My first layer contains the ...
2
votes
2answers
315 views

Combine decision trees from GBM to reduce output

I am curious if any research has been conducted to efficiently combine trees resulting from a gradient boosting process. I routinely run a process that generates 20 or 30 thousand trees in R. I then ...
1
vote
0answers
101 views

How to combine predictions in ensemble

I am trying to learn more about how to build ensembles of predictions in R and coming to a roadblock, and am hoping one can offer guidance. I often read about people automatically identifying how ...
1
vote
0answers
87 views

What are the latest methods to generate ensembles?

I am working with ensembles, and I'm willing to go deep inside the work. I have historical records of: Observations of one variable Historical forecasts for the same variable For future ...
3
votes
2answers
102 views

Combining models for prediction based on residual performance

I have never read or seen someone do this before, so I wanted to pose the question here. Suppose I fit a basic linear model, $\text{price of house} = \beta_0 + \beta_1*\text{taxes} + ...
0
votes
2answers
99 views

Ensemble of models with different feature spaces

BACKGROUND I have data in which the dependent variable is binary with a highly-skewed distribution: <1% records are 1 (doers), >99% records are 0 (non-doers). I'm using logistic regression to ...
2
votes
0answers
68 views

Dissimilarity with earlier features part of cost function

I am using a RandomForest on features (pixels) of images, and I am considering adding cost for "similarity to already other included features" to the cost function. Imagine you have a current RF ...
11
votes
3answers
2k views

Can Random Forest Methodology be Applied to Linear Regressions?

Random Forests work by creating an ensemble of decision trees where each tree is created using a bootstrap sample of the original training data (sample of both input variables and observations). Can ...
4
votes
2answers
207 views

Why are Random Forests splitted based on m random features?

I was watching the following tutorial on Random forests and it says that, "at each node, choose random subset of m features and only consider splitting on those features" I do not understand why we ...