Linked Questions

34
votes
2answers
4k views

Is this the state of art regression methodology?

I've been following Kaggle competitions for a long time and I come to realize that many winning strategies involve using at least one of the "big threes": bagging, boosting and stacking. For ...
6
votes
3answers
5k views

How to combine weak classfiers to get a strong one?

Let as assume that we have a binary classification problem. We also have several classifiers. Instead of assigning a vector to a class (0 or 1) each classifier returns a probability that a given ...
7
votes
1answer
2k views

Main idea of Bagging

I just read this post and several other websites, but I still don't understand what bagging is. I understand it is an algorithm for machine learning, that it improves stability and accuracy of the ...
7
votes
1answer
3k views

Why is tree correlation a problem when working with bagging?

I'm reading ISLR and I don't understand what is the problem that random forests solve; what problems does tree correlation cause when using bagging?
6
votes
2answers
3k views

Bootstrapping test set?

Let's say I have a classification problem with a small and fixed test set. If I train a classifier and report the accuracy on this test set, I know that this estimate has a high variance. Does it make ...
0
votes
3answers
157 views

Why does machine learning work for high-dimensional data($n \ll p$)?

Consider the high dimensional data with which the number of features $p$ is much larger than the number of observations $n$. Machine learning algorithm is trained with the data. My first thought is ...
1
vote
2answers
574 views

How do I optimize decision tree regression algorithm implemented in R?

I'm only getting an accuracy of 59% using the following implementation calculated using the diag(sum(cm)) and sum(cm) functions. ...
3
votes
0answers
432 views

State of the art for bagging/model averaging? [closed]

If I estimate a collection of models predicting $Y$ by $\hat{Y}$, which methods are out there to combine these forecasts? Which methods work well/best (and why?) to improve prediction accuracy? My ...
1
vote
1answer
112 views

Is it ever a good idea to use results of multiple algorithms as features?

Let's say that I have a vector of features that I use to get a single result back using some machine learning algorithm. I thought about using multiple variations of that algorithm to get multiple ...
1
vote
1answer
76 views

Multiple classifiers always wrong one the same examples: could it be exploited?

I trained a different number of classifiers on a problem (logistic regression, KNN, neural networks, svm...). The training happened with double cross validation and all the gold standards, so I'm ...