The tag has no usage guidance.

learn more… | top users | synonyms

0
votes
0answers
9 views

Factoring out the effects of time in a predictive model

I have an ensemble model with a large number of independent variables, and one of those independent variables is time. The data was not generated/measured at regular intervals. There appears to be a ...
0
votes
1answer
23 views

How to embed prior distribution of datasets into the classification model

Given 3 training sets : $(X_1,y_1),(X_2,y_2)$ and $(X_3,y_3)$. These three datasets are separated as it is being manually tagged in the preprocessing. Based on the datasets, three classifiers can be ...
3
votes
1answer
29 views

Stacking: Do more base classifiers always improve accuracy?

When using stacking, can accuracy always be improved by adding more base classifiers, types of base classifiers, and features?
0
votes
1answer
20 views

Chossing an algorithm when there is only one feature

I am combining multiple base classifiers for an ensemble classifier. Different sensors, such as an accelerometer, gyroscope and altimeter are classified individually, and their outputs are then fed ...
1
vote
0answers
20 views

Ensembles - few questions on approach with multiple models

I'm looking for some general-high level understanding of how I should apply ensemble techniques. I understand that some models can be already thought of as ensembles - such as random forests or some ...
1
vote
0answers
33 views

M5 model in ensemble

I am using the M5 model implemented in the RWeka package for predicting a continues variable based on several independent, ecological variables. ...
0
votes
1answer
51 views

How to combine two models (neural network and KNN) in Matlab?

I am conducting research on credit risk prediction using neural networks and K-Nearest Neighbour models (K=10) in Matlab. The dataset has 490 samples with 19 features, from which I want to predict 1 ...
0
votes
1answer
21 views

Combining two or more (SVC) models in Python/scikit.learn

I have some data which I use SVC models with 10 fold cross validation and a parameter grid search on (scikit.learn). I observed that the predictions of some folds have low accuracy, whereas remained ...
0
votes
0answers
52 views

Probability of stacked ensemble classifiers using a simple threshold function

The problem There are N classifiers which each give a probability of some event occurring. We want to combine these probabilities in an ensemble classifier to make a final prediction. We also want to ...
0
votes
1answer
23 views

How are the outcomes that generated from different predictive models combined to get more accurate predictions?

The simple average is commonly used to combine the predictions of different predictive models. Apart form the simple average, what are the other methods that can be used for combining the predictive ...
2
votes
1answer
49 views

Can I use output of classifier A as feature for classifier B?

This is likely to be a confused question, but I'm curious if this is a valid way to combine classifiers. I have a classification data set, i.e. column of labels and N columns of features, and I use a ...
0
votes
1answer
17 views

Is it appropriate to exclude outliers in ensemble learning?

While the term 'appropriate' is usually subjective and largely dependent upon the domain that one is talking about, I am here asking whether the exclusion of outliers goes against the principle of ...
1
vote
0answers
12 views

Understanding Plurality Voting in Ensemble Methods

I'm reading Data mining with decision trees by Rokach, and i've got to a chapter about ensemble methods (using multiple classifiers) and this is where I can't wrap my head around this concept of ...
1
vote
0answers
47 views

Why do boosting overfit on the data with uniform noise?

i read about it but i didn't get the idea, and actually i didn't find many pages that talk about uniform noise with boosting, is it rare to happen or what? another question: i read in some pages that ...
1
vote
0answers
45 views

Isn't stacking models a direct approach to overfitting?

With help by the discussions here I successfully trained various models for classification. As an example say I trained a stochastic gradient boosted model (gbm) and an extreme gradient boosted tree ...
0
votes
0answers
21 views

Combining binary classification algorithms

I have several algorithms which solve a binary classification (with response 0 or 1) problem by assigning to each observation a probability of the target value being equal to 1. All the algorithms try ...
0
votes
1answer
29 views

If you have several models each based on a subset of features, how can you combine the models to make a better prediction?

As a simple example, say you want to predict the house price. You have 5 features. You build 5 models, each trained with one feature. (price, sqft) (price, num_bedrooms) (price, lot_size) (price, ...
0
votes
0answers
12 views

Should we predict residual of ensemble model and add it to final prediction?

We have been doing a time series project on daily wise data. We built 5 different models SVM, XGBoot, ARIMA, KNN, ANN. We then built predictive models for residuals for all of these models and got ...
1
vote
1answer
53 views

Scalable Random Forest For Massive Data

My problem is simple. I want to train a dataset using random forest on a huge dataset (with $n$ rows). Let's assume I can only fit $b < n$ rows in memory at a time. Model Choice I see several ...
1
vote
1answer
18 views

Simple way to combine predictions from multiple classifiers?

I have predictions from 3 binary classifiers (SVM, RF and NN) and would like to combine them in some way. I'm aware of the notion of ensemble learning methods but I was wondering if it would be valid ...
3
votes
1answer
131 views

allocating degree belief to forecast value

We have a number of providers for a forecast of wind power generation per country per date. Values are forecast up to one week ahead. Forecasts may be compared with actual values of reported wind ...
1
vote
1answer
23 views

Trees of ensembles.

I have a large dataset (100k+), and it's growing everyday. I want to train it to predict a value (a regression problem). I've been finding that ensemble trees work the best for now, but in the ...
0
votes
0answers
36 views

How do they do classifier ensemble using stacking

I am very new in data science, now I am trying to learn Ensembling with Azure Machine Learning. This example seems interesting Ensemble Classifier using Stacking But when I checked those modules in ...
0
votes
0answers
13 views

How can one quantify the variable importance dilution effect in random forests (and similar statistical learning methods)?

In Applied Predictive Modelling (Kuhn, Johnson, 2013, p 202), the authors refer to a dilution effect whereby compared to a single tree or a classical regression technique, the difference in importance ...
1
vote
1answer
63 views

Why are gradient boosting regression trees good candidates for ranking problems?

I have been reading up on gradient boosting machines, and in particular GBRT's. I've come across numerous mentions (and finally tracked down some papers) on applying these models to ranking problems - ...
0
votes
0answers
15 views

kNN used as a metafeature for future ensemble

Lately I've saw a lot of classification approaches for large datasets that involved ensemble methods, most of them using kNN. If I don't miss-understand the algorithm, it is meant to use a portion of ...
2
votes
1answer
77 views

Ensemble models perform worse than single one

In my model testing, I tried to use model ensembling (blending in this case) to get better results. However the ensemble cannot beat single RandomForrestClassifier. In first layer, I train meta-...
0
votes
0answers
12 views

Can I avoid overfit in an ensemble model using a bootstrap with small samples as training sets?

Let´s suppose that t I have a dataset with 250 data points and I want to train an ensemble. If I choose to train each classifier of the ensemble with a small bootstrap sample (10) of the original ...
1
vote
1answer
57 views

How to proceed with building an ensemble classifier using Naive Bayes, TAN and Logistic Regression in R

I'm relatively new to machine learning (started about 5 months ago), and I'm looking at potentially implementing an ensemble classifier as part of my research. I have built 3 models that I use to ...
1
vote
2answers
61 views

Ensemble model of simple univariate linear regression models

Calculating univariate linear regression and correlation is simple in SQL. It can be done like so: ...
2
votes
0answers
19 views

Multiclass classification one versus one with ensemble

If I use an ensemble, which consists of four classifiers, in order to classify my data into three classes. Further, suppose I use the one versus one strategy. My question is: How to fuse the ...
0
votes
0answers
13 views

Combining classification results

Let's say I have a binary classification problem and several models $M_1, \dots, M_n$ to predict the classification results. In my setting the outputs of these models are quantitative (some of them ...
1
vote
0answers
52 views

ensemble methods:voting with average of probabilities in weka

output attribute is risky patient. Values are yes and no.if yes then patient is risky and if no then patient is not risky. If I am combining 3 classifier for classification model in weka, and if ...
0
votes
1answer
17 views

Weighing classifiers based on the cross validation accuracy

Suppose there exist n individual classifiers which have different parameters and have been trained on the same data. In order to build an ensemble of these classifiers, is it an optimal method to ...
0
votes
0answers
51 views

How to evaluate stacking ensemble model vs. other models with 10-fold cross-validation?

I've been comparing various predictive models for both continuous and binary outcomes for a health care model. So far 10-fold cross-validation has been useful: training models on 9/10 of the analysis ...
1
vote
0answers
27 views

Model selection in ensembles

I'm trying to build an ensemble for a ML problem where fast prediction time is critical. So I'm interested in keeping my set of level-0 models for the ensemble pruned. Which measures can I use to ...
1
vote
0answers
36 views

Boosting in unsupervised learning - methods and use cases

I'm looking for methods and uses cases for applying boosting or other ensemble methods for unsupervised learning Examples of such methods and use cases are: Boosting density estimation Saharon ...
3
votes
0answers
18 views

Theory of correlation and weighing when ensabling models

I'm ensabling models together to improve the overall performance. At the moment, I'm weighing each model by its performance under cross-validation, and this works reasonably well. Clearly the best ...
0
votes
0answers
14 views

Classification: estimate how many people are in a household through account transactions

I want to estimate how many people are in a household by looking at account transactions. It would be also interesting to understand if in the household there are children. I thought that a possible ...
3
votes
1answer
154 views

stacking and blending of regression models

I am self-studying blending and stacking, and am especially interested in this in the context of regression models. I have been reading a number of the stacking, blending and bagging links posted on ...
0
votes
0answers
29 views

Combining strong and weak learners

Our NGO is struggling with the following problem: For an ecological application, we have tried to program a deep neural net to estimate the size and weight of images of birds that we have captured on ...
0
votes
0answers
17 views

How to decide about the number of looks (window size for ensemble averaging) in SAR images?

This question has frustrated me for a while. In order to find an answer I sent an email to prof. Yamaguchi, the author of the paper Four-Component Scattering Power Decomposition With Rotation of ...
1
vote
0answers
44 views

simple statistics of aggregated posterior data after ensemble data assimilation

I have $N$ "4-dimensional" arrays $(x,y,t,c)_{i=1:N}$ containing greenhouse gas emission data, where $(x,y)$ are the spatial coordinates $t$ is the time coordinate (discrete time points) $c$ is the ...
1
vote
1answer
30 views

Combine multiple predictions of binary outcome

I am moving from a single-model prediction of a binary outcome to an aggregate of a small number of models, for example: ...
2
votes
0answers
108 views

Combining multiple classifiers

I am trying to do a binary classification of text articles into {relevant, non-relevant}. The text articles have following features: [[article text, ...
3
votes
0answers
103 views

How to combine regression models?

Say I have three data sets of size $n$ each: $y_1$ = heights of people from the US only $y_2$ = heights of men from the whole world $y_3$ = heights of women from the whole world And I build a ...
1
vote
0answers
103 views

Ensembling with VotingClassifier

I am using VotingClassifier from sklearn.ensemble however i am puzzled with the results. Consider following algorithms: ...
1
vote
0answers
32 views

Reduction of accuracy - Bagging

Bagging is an ensemble method which uses a parallel set of classifiers and then gives consensus output. Usually bagging improves the accuracy of a classification. But if there are situations in which ...
3
votes
1answer
63 views

What happens when Bagging does not have a majority vote?

I have a question regarding the bagging technique used in ensemble learning. Let's assume I have 6 classifiers which could classify a response variable which has 3 finite categories(...
8
votes
1answer
186 views

Gradient Boosting for Linear Regression - why does it not work?

While learning about Gradient Boosting, I haven't heard about any constraints regarding the properties of a "weak classifier" that the method uses to build and ensemble model. However, I could not ...