Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms (1)

160
votes
15answers
35k views

The Two Cultures: statistics vs. machine learning?

Last year, I read a blog post from Brendan O'Connor entitled "Statistics vs. Machine Learning, fight!" that discussed some of the differences between the two fields. Andrew Gelman responded favorably ...
91
votes
8answers
13k views

Detecting a given face in a database of facial images

I'm working on a little project involving the faces of twitter users via their profile pictures. A problem I've encountered is that after I filter out all but the images that are clear portrait ...
71
votes
12answers
43k views

What is the difference between data mining, statistics, machine learning and AI?

What is the difference between data mining, statistics, machine learning and AI? Would it be accurate to say that they are 4 fields attempting to solve very similar problems but with different ...
67
votes
5answers
7k views

What skills are required to perform large scale statistical analyses?

Many statistical jobs ask for experience with large scale data. What are the sorts of statistical and computational skills that would be need for working with large data sets. For example, how about ...
64
votes
6answers
7k views

Why is Euclidean distance not a good metric in high dimensions?

I read that 'Euclidean distance is not a good distance in high dimensions'. I guess this statement has something to do with the curse of dimensionality, but what exactly? Besides, what is 'high ...
49
votes
8answers
7k views

Skills hard to find in machine learners?

It seems that data mining and machine learning became so popular that now almost every CS student knows about classifiers, clustering, statistical NLP ... etc. So it seems that finding data miners is ...
49
votes
1answer
2k views

Help me understand Support Vector Machines

I understand the basics of what a Support Vector Machines' aim is in terms of classifying an input set into several different classes, but what I don't understand is some of the nitty-gritty details. ...
48
votes
6answers
32k views

What is the difference between test set and validation set?

I found this confusing when I use the neural network toolbox in Matlab. It divided the raw data set into three parts: training set validation set test set I notice in many training or learning ...
47
votes
9answers
20k views

Having a job in data-mining without a PhD

I've been very interested in data-mining and machine-learning for a while, partly because I majored in that area at school, but also because I am truly much more excited trying to solve problems that ...
39
votes
7answers
11k views

Machine Learning using Python

I am considering using Python libraries for doing my Machine Learning experiments. Thus far, I had been relying on WEKA but have been pretty dissatisfied on the whole. This is primarily because I have ...
39
votes
1answer
9k views

Conditional inference trees vs traditional decision trees

Can anyone explain the primary differences between conditional inference trees (ctree from party package in R) compared to the ...
36
votes
12answers
5k views

Machine learning cookbook / reference card / cheatsheet?

I find resources like the Probability and Statistics Cookbook and The R Reference Card for Data Mining incredibly useful. They obviously serve well as references but also help me to organize my ...
35
votes
5answers
5k views

Training with the full dataset after cross-validation?

Is it always a good idea to train with the full dataset after k-fold cross-validation? Or is it better instead to stick with one of the models learned in one of the cross-validation splits for ...
35
votes
4answers
11k views

What does the hidden layer in a neural network compute?

I'm sure many people will respond with links to 'let me google that for you', so I want to say that I've tried to figure this out so please forgive my lack of understanding here, but I cannot figure ...
33
votes
3answers
14k views

Best way to present a random forest in a publication?

I am using the random forest algorithm as a robust classifier of two groups in a microarray study with 1000s of features. What is the best way to present the random forest so that there is enough ...
30
votes
8answers
4k views

Feature selection for “final” model when performing cross-validation in machine learning

I am getting a bit confused about feature selection and machine learning and I was wondering if you could help me out. I have a microarray dataset that is classified into two groups and has 1000s of ...
30
votes
1answer
5k views

ROC vs precision-and-recall curves

I understand the formal differences between them, what I want to know is when it is more relevant to use one vs. the other. Do they always provide complementary insight about the performance of a ...
30
votes
4answers
9k views

How to compute precision/recall for multiclass-multilabel classification?

I'm wondering how to calculate precision and recall measures for multiclass multilabel classification, i.e. classification where there are more than two labels, and where each instance can have ...
29
votes
4answers
8k views

How to select kernel for SVM?

When using SVM, we need to select a kernel. I wonder how to select a kernel. Any criteria on kernel selection?
29
votes
3answers
657 views

Application of machine learning methods in StackExchange websites

I have a Machine Learning course this semester and the professor asked us to find a real-world problem and solve it by one of machine learning methods introduced in the class, as: Decision Trees ...
28
votes
8answers
1k views

How can I help ensure testing data does not leak into training data?

Suppose we have someone building a predictive model, but that someone is not necessarily well-versed in proper statistical or machine learning principles. Maybe we are helping that person as they are ...
28
votes
4answers
2k views

Cloud computing platforms for machine learning

I've got a small list of companies that provide a platform for running R, python, or octave scripts on clusters built on top of amazon EC2. Are there other names I should add? Cloudnumbers Opani ...
27
votes
2answers
6k views

Choice of K in K-Fold cross validation

I've been using the K-Fold cross validation a few times now to evaluate performance of some learning algorithms, but I've always been puzzled as to how I should choose the value of K. I've often seen ...
25
votes
5answers
3k views

Is machine learning less useful for understanding causality, thus less interesting for social science?

My understanding of the difference between machine learning/other statistical predictive techniques vs. the kind of statistics that social scientists (e.g., economists) use is that economists seem ...
24
votes
1answer
3k views

What is behind Google Prediction API?

Google Prediction API is a cloud service where user can submit some training data to train some mysterious classifier and later ask it to classify incoming data, for instance to implement spam filters ...
23
votes
1answer
6k views

Bagging, boosting and stacking in machine learning

What's the similarities and differences beetween this 3 methods: bagging, boosting, stacking? Which is the best one? And why? Can you give me an example for each?
23
votes
2answers
3k views

Generative vs. discriminative

I know that generative means "based on $P(x,y)$" and discriminative means "based on $P(y|x)$," but I'm confused on several points: Wikipedia (+ many other hits on the web) classify things like SVMs ...
23
votes
2answers
4k views

How does a Support Vector Machine (SVM) work?

How does a Support Vector Machine (SVM) work, and what differentiates it from other linear classifiers, such as the Linear Perceptron, Linear Discriminant Analysis, or Logistic Regression? * (* I'm ...
22
votes
6answers
4k views

Can you recommend a book to read before Elements of Statistical Learning?

Based on this post, http://quant.stackexchange.com/questions/111/how-can-i-go-about-applying-machine-learning-algorithms-to-stock-markets, I want to digest Elements of Statistical Learning. ...
22
votes
6answers
2k views

Data mining: How should I go about finding the functional form?

I'm curious about repeatable procedures that can be used to discover the functional form of the function y = f(A, B, C) + error_term where my only input is a set of ...
22
votes
3answers
2k views

Understanding Naive Bayes

To demonstrate the concept of Naïve Bayes Classification, consider the example displayed in the illustration above. As indicated, the objects can be classified as either GREEN or RED. My task is to ...
22
votes
2answers
4k views

Proper way of using recurrent neural network for time series analysis

Recurrent neural networks differ from "regular" ones by the fact that they have a "memory" layer. Due to this layer, recurrent NN's are supposed to be useful in time series modelling. However, I'm not ...
21
votes
5answers
1k views

Can you overfit by training machine learning algorithms using CV/Bootstrap?

This question may well be too open ended to get a definitive answer, but hopefully not. Machine learning algorithms, such as SVM, GBM, Random Forest etc, generally have some free parameters that, ...
21
votes
3answers
695 views

Why is there an asymmetry between the training step and evaluation step?

It is well-known, especially in natural language processing, that machine learning should proceed in two steps, a training step and an evaluation step, and they should use different data. Why is ...
21
votes
3answers
14k views

How to calculate precision and recall for multiclass classification using confusion matrix?

I wonder how to compute precision and recall using a confusion matrix for a multi-class classification problem. In specific, one observation can only be assigned with most probable class / label. I ...
21
votes
3answers
6k views

Using deep learning for time series prediction

I'm new in area of deep learning and for me first step was to read interesting articles from deeplearning.net site. In papers about deep learning, Hinton and others mostly talk about applying it to ...
21
votes
3answers
3k views

How well does R scale to text classification tasks?

I am trying to get upto speed with R. I eventually want to use R libraries for doing text classification. I was just wondering what people's experiences are with regard to R's scalability when it ...
20
votes
3answers
13k views

Things to consider about masters programs in statistics

It is admission season for graduate schools. I (and many students like me) am now trying to decide which statistics program to pick. What are some things those of you who work with statistics ...
20
votes
5answers
5k views

Free data set for very high dimensional classification

What are the freely available data set for classification with more than 1000 features (or sample points if it contains curves)? There is already a community wiki about free data sets: ...
20
votes
3answers
772 views

First step for big data ($N = 10^{10}$, $p = 2000$)

Suppose you are analyzing a huge data set at the tune of billions of observations per day, where each observation has a couple thousand sparse and possibly redundant numerical and categorial ...
19
votes
4answers
391 views

Addressing model uncertainty

I was wondering how the Bayesians in the CrossValidated community view the problem of model uncertainty and how they prefer to deal with it? I will try to pose my question in two parts: How ...
19
votes
3answers
1k views

Is it important for statisticians to learn machine learning?

Is machine learning an important subject for any statistician to become acquainted with? It seems that machine learning is statistics. Why don't statistics programs (undergraduate and graduate) ...
19
votes
2answers
7k views

What are the main differences between K-means and K-nearest neighbours?

I know that k-means is unsupervised and is used for clustering etc and that k-NN is supervised. But I wanted to know concrete differences between the two?
19
votes
3answers
5k views

Creating a “certainty score” from the votes in random forests?

I am looking to train a classifier that will discriminate between Type A and Type B objects with a reasonably large training set ...
19
votes
2answers
756 views

How to choose between learning algorithms

I need to implement a program that will classify records into 2 categories (true/false) based on some training data, and I was wondering at which algorithm/methodology I should be looking at. There ...
19
votes
4answers
765 views

What exactly is a Bayesian model?

Can I call a model wherein Bayes' Theorem is used a "Bayesian model"? I am afraid such a definition might be too broad. So what exactly is a Bayesian model?
19
votes
2answers
2k views

Sites for predictive modeling competitions

I participate in predictive modeling competitions on Kaggle, TunedIt, and CrowdAnalytix. I find that these sites are a good way to "work-out" for statistics/machine learning. Are there any other ...
18
votes
6answers
3k views

Variable selection procedure for binary classification

What are the variable/feature selection that you prefer for binary classification when there are many more variables/feature than observations in the learning set? The aim here is to discuss what is ...
18
votes
4answers
468 views

To what extent is the distinction between correlation and causation relevant to Google?

Context A popular question on this site is " What are common statistical sins?". One of the sins mentioned is assuming that "correlation implies causation..." link Then, in the comments with 5 ...
17
votes
2answers
4k views

Feed-forward and recurrent neural networks?

What is the difference between a feed-forward and recurrent neural network? Why would you use one over the other? Do other network topologies exist?