Statistical classification is the problem of identifying the sub-population to which new observations belong, where the identity of the sub-population is unknown, on the basis of a training set of data containing observations whose sub-population is known. Therefore these classifications will show a ...
36
votes
1answer
1k views
Help me understand Support Vector Machines
I understand the basics of what a Support Vector Machines' aim is in terms of classifying an input set into several different classes, but what I don't understand is some of the nitty-gritty details. ...
26
votes
5answers
2k views
Alternatives to logistic regression in R
I would like as many algorithms that perform the same task as logistic regression. That is algorithms/models that can give a prediction to a binary response (Y) with some explanatory variable (X).
...
23
votes
6answers
2k views
Feature selection for “final” model when performing cross-validation in machine learning
I am getting a bit confused about feature selection and machine learning
and I was wondering if you could help me out. I have a microarray dataset that is
classified into two groups and has 1000s of ...
22
votes
6answers
878 views
Statistical classification of text
I'm a programmer without statistical background, and I'm currently looking at different classification methods for a large number of different documents that I want to classify into pre-defined ...
22
votes
3answers
7k views
Best way to present a random forest in a publication?
I am using the random forest algorithm as a robust classifier of two groups in a microarray study with 1000s of features.
What is the best way to present the random forest so that there is enough ...
20
votes
2answers
577 views
Detecting patterns of cheating on a multi-question exam
QUESTION:
I have binary data on exam questions (correct/incorrect). Some individuals might have had prior access to a subset of questions and their correct answers. I don’t know who, how many, or ...
19
votes
4answers
421 views
Why does the least square solution give poor results in this case?
There is an image in page 204, chapter 4 of "pattern recognition and machine learning" by Bishop where I don't understand why the Least square solution gives poor results here:
The previous ...
18
votes
7answers
461 views
How can I help ensure testing data does not leak into training data?
Suppose we have someone building a predictive model, but that someone is not necessarily well-versed in proper statistical or machine learning principles. Maybe we are helping that person as they are ...
18
votes
3answers
5k views
How to compute precision/recall for multiclass-multilabel classification?
I'm wondering how to calculate precision and recall measures for multiclass multilabel classification, i.e. classification where there are more than two labels, and where each instance can have ...
17
votes
3answers
890 views
Why do naive Bayesian classifiers perform so well?
Naive Bayes classifiers are a popular choice for classification problems. There are many reasons for this, including:
"Zeitgeist" - widespread awareness after the success of spam filters about ten ...
16
votes
2answers
795 views
How does a Support Vector Machine (SVM) work?
How does a Support Vector Machine (SVM) work, and what differentiates it from other linear classifiers, such as the Linear Perceptron, Linear Discriminant Analysis, or Logistic Regression? *
(* I'm ...
15
votes
2answers
2k views
Choice of K in K-Fold cross validation
I've been using the K-Fold cross validation a few times now to evaluate performance of some learning algorithms, but I've always been puzzled as to how I should choose the value of K.
I've often seen ...
15
votes
1answer
206 views
I want to build a crime index and political instability index based in news stories
I have this side project where I crawl the local news websites in my country and want to build a crime index and political instability index.
I have already covered the information retrieval part of ...
14
votes
4answers
5k views
How to plot ROC curves in multiclass classification?
In other words, instead of having a two class problem I am dealing with 4 classes and still would like to assess performance using AUC.
14
votes
4answers
67 views
Classification with tall fat data
I need to train a linear classifier on my laptop with hundreds of thousands of data points and about ten thousand features. What are my options? What is the state of the art for this type of problem?
...
14
votes
2answers
331 views
Combining classifiers by flipping a coin
I am studying a machine learning course and the lecture slides contain information what I find contradicting with the recommended book.
The problem is the following: there are three classifiers:
...
13
votes
10answers
586 views
What is the best out-of-the-box 2-class classifier for your application?
Rules:
one classifier per answer
vote up if you agree
downvote/remove duplicates.
put your application in the comment
13
votes
5answers
972 views
Alternatives to classification trees, with better predictive (e.g: CV) performance?
I am looking for an alternative to Classification Trees which might yield better predictive power.
The data I am dealing with has factors for both the explanatory and the explained variables.
I ...
13
votes
1answer
177 views
Random forest assumptions
I am kind of new to random forest so I am still struggling with some basic concepts.
In linear regression, we assume independent observations, constant variance…
What are the basic ...
13
votes
5answers
2k views
Free data set for very high dimensional classification
What are the freely available data set for classification with more than 1000 features (or sample points if it contains curves)?
There is already a community wiki about free data sets:
...
12
votes
6answers
1k views
Variable selection procedure for binary classification
What are the variable/feature selection that you prefer for binary classification when there are many more variables/feature than observations in the learning set? The aim here is to discuss what is ...
12
votes
2answers
1k views
Restricted Boltzmann machines vs multilayer neural networks
I've been wanting to experiment with a neural network for a classification problem that I'm facing. I ran into papers that talk of RBMs. But from what I can understand, they are no different from ...
12
votes
4answers
713 views
What is a good resource that includes a comparison of the pros and cons of different classifiers?
What is the best out-of-the-box 2-class classifier? Yes, I guess that's the million dollar question, and yes, I'm aware of the no free lunch theorem, and I've also read the previous questions:
...
12
votes
3answers
2k views
Top five classifiers to try first
Besides obvious classifier characteristics like
computational cost,
expected data types of features/labels and
suitability for certain sizes and dimensions of data sets,
what are the top five (or ...
12
votes
3answers
182 views
Is building a multiclass classifier better than several binary ones?
I need to classify URLs into categories. Say I have 15 categories that I'm planning to zero down every URL to.
Is a 15-way classifier better? Where I have 15 labels and generate features for each ...
12
votes
1answer
403 views
Reproducing table 18.1 from “Elements of Statistical Learning”
Table 18.1 in the Elements of Statistical Learning summarizes the performance of several classifiers on a 14 class data set. I am comparing a new algorithm with the lasso and elastic net for such ...
11
votes
4answers
714 views
Statistical similarity of time series
Supposing one has a time series from which one can take various measurements such as period, maximum, minimum, average etc. and then use these to create a model sine wave with the same attributes, are ...
11
votes
9answers
3k views
Social network datasets
I am looking for social network datasets (twitter, friendfeed, facebook, lastfm, etc.) for classification tasks, preferibly in arff format.
My searches at UCI and Google wasn't successful so far.. ...
11
votes
2answers
233 views
Summary of “Large p, Small n” results
Can anybody point me to a survey paper on "Large $p$, Small $n$" results? I am interested in how this problem manifests itself in different research contexts, e.g. regression, classification, ...
10
votes
5answers
358 views
What's the correct way to test the significance of classification results
There are many situations where you may train several different classifiers, or use several different feature extraction methods. In the literature authors often give the mean classification error ...
10
votes
3answers
540 views
Classification model for movie rating prediction
I am somewhat new to data mining, and I am working on a classification model for movie rating prediction.
I have collected data sets from IMDB, and I am planning to use a decision trees and nearest ...
10
votes
1answer
427 views
Is cross validation a proper substitute for validation set?
In text classification, I have a training set with about 800 samples, and a test set with about 150 samples. The test set has never been used, and waiting to be used until the end.
I am using the ...
10
votes
1answer
305 views
Test for proportions and binary classifier
I have a prototype machine producing parts.
In a first test the machine produces $N_1$ parts and a binary classifier tells me that $d_1$ parts are defective ($d_1 < N_1$, usually $d_1/N_1<0.01$ ...
9
votes
4answers
50 views
Possible to get a better ANN by removing some connections?
I was wondering if there under some circumstances is possible for ANN's to perform better if you prune away some connections on them as for example:
Constructing one ANN by taking two multi-layered ...
9
votes
2answers
439 views
Why not approach classification through regression?
Some material I've seen on machine learning said that it's a bad idea to approach a classification problem through regression. But I think it's always possible to do a continuous regression to fit the ...
9
votes
4answers
366 views
Which statistical classification algorithm can predict true/false for a sequence of inputs?
Given a sequence of inputs, I need to determine whether this sequence has a certain desired property. The property can only be true or false, that is, there are only two possible classes that a ...
9
votes
2answers
3k views
How to produce a pretty plot of the results of k-means cluster analysis?
I'm using R to do K-means clustering. I'm using 14 variables to run K-means
What is a pretty way to plot the results of K-means?
Are there any existing implementations?
Does having 14 variables ...
9
votes
3answers
633 views
When are Shao's results on leave-one-out cross-validation applicable?
In his paper Linear Model Selection by Cross-Validation, Jun Shao shows that for the problem of variable selection in multivariate linear regression, the method of leave-one-out cross validation ...
9
votes
1answer
129 views
Does a sparse training set adversely affect an SVM?
I'm trying to classify messages into different categories using an SVM. I've compiled a list of desirable words/symbols from the training set.
For each vector, which represents a message, I set the ...
8
votes
4answers
645 views
Why do researchers use 10-fold cross validation instead of testing on a validation set?
I have read a lot of research papers about sentiment classification and related topics.
Most of them use 10-fold cross validation to train and test classifiers. That means that no separate ...
8
votes
2answers
410 views
Can CART models be made robust?
A colleague in my office said to me today "Tree models aren't good because they get caught by extreme observations".
A search here resulted in this thread that basically supports the claim.
Which ...
8
votes
1answer
207 views
Should I re-shuffle my data?
We have a set biological samples that was pretty expensive to obtain. We put these samples through a series of tests to generate data that are used for building a predictive model. For this purpose we ...
8
votes
2answers
360 views
Applying machine learning for DDoS filtering
In Stanford's Machine Learning course Andrew Ng mentioned applying ML in IT.
Some time later when I got moderate size(about 20k bots) DDoS on our site I decided to fight against it using simple Neural ...
8
votes
5answers
2k views
Large scale text classification
I am looking to do classification on my text data. I have 300 classes, 200 training documents per class (so ...
8
votes
3answers
144 views
Detecting clusters in a binary sequence
I have a binary sequence such as 11111011011110101100000000000100101011011111101111100000000000011010100000010000000011101111
Where clusters of mostly 1's are ...
8
votes
1answer
201 views
Stratified classification with random forests (or another classifier)
So, I've got a matrix of about 60 x 1000. I'm looking at it as 60 objects with 1000 features; the 60 objects are grouped into 3 classes (a,b,c). 20 objects in each class, and we know the true ...
8
votes
2answers
756 views
How to understand a convolutional deep belief network for audio classification?
In "Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations" by Lee et. al.(PDF) Convolutional DBN's are proposed. Also the method is evaluated for image ...
8
votes
1answer
128 views
How to handle the difference between the distribution of the test set and the training set?
I think one basic assumption of machine learning or parameter estimation is that the unseen data come from the same distribution as the training set. However, in some practical cases, the distribution ...
8
votes
1answer
1k views
RandomForest - MDS plot interpretation
I used randomForest to classify 6 animal behaviours (eg. Standing, Walking, Swimming etc.) based on 8 variables (different body postures and movement).
The MDSplot in the randomForest package gives ...
8
votes
1answer
175 views
Categorization/Segmentation techniques
First, let me say that I am a bit out of my depth here, so if this question needs to be re-phrased or closed as a duplicate, please let me know. It may simply be that I don't have the proper ...
