Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms (1)

6
votes
1answer
283 views

Probabilistic outputs from SVMs

I remember a paper from 1999 (13 years ago!) called Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods (1999) by John Platt that outlined a method for ...
5
votes
4answers
3k views

Fastest SVM implementation

More of a general question. I'm running a rbf SVM for predictive modeling. I think my current program definitely needs a bit of a speed up. I use scikits learn with a coarse to fine grid search + ...
7
votes
1answer
83 views

Classification of conversations based on content

I'd like to be able to design a classifier that can distinguish between different types of conversations (not necessarily tell anything about mood, sincerity, or outcome, that's a bit too far ...
2
votes
0answers
306 views

Mackey-glass regression experiments

So I'm running some experiments using long-short term memory networks for temporal sequence regression, and one of the sequences I have to use is generated using the mackey-glass differential ...
8
votes
1answer
585 views

Dealing with very large time-series datasets

I have access to a very large dataset. The data is from MEG recordings of people listening to musical excerpts, from one of four genres. The data is as follows: 6 Subjects 3 Experimental repetitions ...
8
votes
1answer
136 views

Beyond Fisher kernels

For a while, it seemed like Fisher Kernels might become popular, as they seemed to be a way to construct kernels from probabilistic models. However, I've rarely seen them used in practice, and I have ...
7
votes
2answers
195 views

What is the Drosophila of AI now?

In the mid 1960s, researchers have famously referred to chess as the "drosophila of AI": like the fruit fly, the game of chess was accessible and relatively simple problem to experiment on, which yet ...
2
votes
0answers
85 views

Dynamic recommender systems [duplicate]

Possible Duplicate: Dynamic recommender systems A Recommender System would measure the correlation between ratings of different users and yield recommendations for a given user about the ...
3
votes
2answers
425 views

When parsing text for n-grams - should punctuation be included?

I want to start working on data-mining by parsing text. It seems like the best place to start is by processing n-grams out of text to try sentiment analysis. ...
13
votes
2answers
1k views

Best bandit algorithm?

The most well-known bandit algorithm is upper confidence bound (UCB) which popularized this class of algorithms. Since then I presume there are now better algorithms. What is the current best ...
11
votes
2answers
2k views

In boosting, why are the learners “weak”?

See also a similar question on stats.SE. In boosting algorithms such as AdaBoost and LPBoost it is known that the "weak" learners to be combined only have to perform better than chance to be useful, ...
27
votes
2answers
6k views

How does a Support Vector Machine (SVM) work?

How does a Support Vector Machine (SVM) work, and what differentiates it from other linear classifiers, such as the Linear Perceptron, Linear Discriminant Analysis, or Logistic Regression? * (* I'm ...
8
votes
3answers
288 views

Learning on huge datasets

Basically, there are two common ways to learn against huge datasets (when you're confronted by time/space restrictions): Cheating :) - use just a "manageable" subset for training. The loss of ...
12
votes
2answers
2k views

How does linear discriminant analysis reduce the dimensions?

There are words from "The Elements of Statistical Learning" on page 91: The K centroids in p-dimensional input space span at most K-1 dimensional subspace, and if p is much larger than K, this ...
2
votes
0answers
169 views

Formula for marginal probability in CRF++

On the website for CRF++ http://crfpp.sourceforge.net/ they mention that marginal probabilities can be output for each possible label. My question is, in CRF theory, what's the formula for this ...
0
votes
0answers
89 views

Computing overhead of statistical models for training?

Could someone provide overhead of the following model for training (With respect to input size or if there are any relevant parameters). Overhead I mean somewhat like asymptotic time complexity form. ...
3
votes
1answer
80 views

Fiducial Inference in Machine Learning

I was looking at the Fiducial Inference page on wikipedia, which is an alternative to the traditional Frequentist and Bayesian standpoints. Although it was out of favour in mainstream statistics for ...
8
votes
1answer
268 views

Are MFCCs the optimal method of representing music to a retrieval system?

A signal processing technique, the Mel frequency Cepstrum, is often used to extract information from a musical piece for use in a machine learning task. This method gives a short-term power spectrum, ...
16
votes
4answers
571 views

How would you design a machine learning system to play Angry Birds?

After playing way too much Angry Birds, I started to observe my own strategies. It turns out that I developed a very specific approach to getting 3 stars on each level. That made me wonder about the ...
2
votes
1answer
263 views

Text mining “power calculations”?

I have a list of 6,500 or so medical treatments, of which I have classified 700 or so as involving a physician or not. I am interested in both the specific question of how to calculate whether 700 ...
1
vote
1answer
44 views

Learning association between multiple effects and causal outcomes

We have some data represented as follows: on each row there are 10 observable effects, and 3 non-observable causes out of 8 that were guessed by scholars. Effects are real numbers, and causes are ...
1
vote
1answer
297 views

Is the true relation between independent and dependent variables assumed to be a function or a distribution?

In classification and regression tasks, we try to learn from a training data set a function mapping a independent variable $X$ to a dependent variable $Y$. When evaluating the error rate of a ...
11
votes
1answer
863 views

Understanding no free lunch theorem in Duda et al's Pattern Classification

I have some questions about the notations used in Section 9.2 Lack of Inherent Superiority of Any Classifier in Duda, Hart and Stork's Pattern Classification. First let me quote some relevant text ...
3
votes
1answer
275 views

Multiclass SVM + Ineffective X Validation, Time Series Prediction

I've recently run into an interesting and rather odd problem with cross validating a multiclass SVM that I can't figure out. Basically, I have a timeseries to predict and have created a dataset of ...
14
votes
2answers
1k views

Why does Natural Language Processing not fall under Machine Learning domain? [closed]

I encounter it in many books as well as web. Natural Language Processing and Machine Learning are said to be different subsets of Artificial Intelligence. Why is it? We can achieve results of Natural ...
8
votes
1answer
2k views

How does the Kinect use random forests?

I read on this site that apparently the Kinect uses the random forests algorithm for machine learning in some way. Can anyone explain what it uses random forests for, and how their approach works?
9
votes
2answers
557 views

Dealing with datasets with a variable number of features

What are some approaches for classifying data with a variable number of features? As an example, consider a problem where each data point is a vector of x and y points, and we don't have the same ...
16
votes
2answers
712 views

Are machine learning techniques “approximation algorithms”?

Recently there was a ML-like question over on cstheory stackexchange, and I posted an answer recommending Powell's method, gradient descent, genetic algorithms, or other "approximation algorithms". In ...
15
votes
7answers
2k views

What do statisticians do that can't be automated?

Will software eventually make statisticians obsolete? What is done that can't be programmed into a computer?
2
votes
0answers
621 views

Cross validation procedure - is this right?

Just want to check that I am performing my cross validation procedures right. I'm using a non-linear svm. I do a five fold cross validation (5 splits of test/train on my original training data) and ...
0
votes
1answer
147 views

Any advice on technical analysis of financial data?

we're looking for any software/framework that is able to analyse the financial flux? Some preliminaries. We've been asked by one of our customer to find or develop the software that would solve some ...
1
vote
0answers
59 views

Functional dependency in Support Vector Machine input

I'm currently training a support vector machine with a second order Markov transition matrix. These probabilities are taken from a computer image, and are the probabilities of adjacent pixels having ...
12
votes
3answers
1k views

What can one tell a school kid about statistics and machine learning?

Next week we have an intern from a local school in the house. The concept behind his short internship is to get an idea how the real world works and what certain jobs deal with, how the daily work ...
2
votes
1answer
175 views

SVM and cross validation with a minimum finding algorithm

Just a simple question on parameter selection for SVMs. If I use a minimum finding algorithm to find the optimal parameters for a set of data, how do I "average" the parameters over a set of cross ...
2
votes
2answers
2k views

SVM parameter selection and cross validation

Have a quick question about parameter selection for an SVM. I'm using a rbf kernel, so trying to optimize C and gamma. I have an example set of around 4500, about 700 features, and using 700 examples ...
7
votes
1answer
340 views

Creating a maximum entropy Markov model from an existing multi-input maximum entropy classifier

I am intrigued by the concept of a Maximum Entropy Markov Model (MEMM), and I am thinking of using it for a Part of Speech (POS) tagger. At the moment, I am using a conventional Maximum Entropy (ME) ...
11
votes
1answer
523 views

Predictive Modeling - Should we care about mixed modeling?

For predictive modeling, do we need to concern ourselves with statistical concepts such as random effects and non independence of observations (repeated measures)? For example.... I have data from 5 ...
23
votes
3answers
820 views

Why is there an asymmetry between the training step and evaluation step?

It is well-known, especially in natural language processing, that machine learning should proceed in two steps, a training step and an evaluation step, and they should use different data. Why is ...
1
vote
0answers
509 views

Performance of the fuzzy c-means clustering algorithm

I have implemented a genetic algorithm for a fuzzy c-means clustering in Matlab. Its performance should be apriori better than that of the classic fuzzy c-means (fcm function in matlab). However, on ...
12
votes
2answers
431 views

Updating classification probability in logistic regression through time

I am building a predictive model that forecasts a student's probability of success at the end of a term. I’m specifically interested in whether the student succeeds or fails, where success is usually ...
1
vote
2answers
513 views

How to decrease training set size?

I have a large training set, and it is too big to apply some algorithms due to computation limits. What are the common methods to decrease training set size without losing significant amount of ...
13
votes
2answers
3k views

Why not approach classification through regression?

Some material I've seen on machine learning said that it's a bad idea to approach a classification problem through regression. But I think it's always possible to do a continuous regression to fit the ...
0
votes
1answer
156 views

Measuring information content of a random variable in Naive Bayes classifier

I'm trying to improve accuracy in a Naive Bayes classifier that uses a bunch of features. I have a hunch that removing some features may actually improve performance. My reasoning is for a ...
1
vote
1answer
90 views

Need help in automatically computing optimal hyper-parameters for a clustering algorithm

I have an algorithm that take as input some data (that are continuously arriving) and 3 or 4 parameter values that should be specified by the user. At the and of execution (or periodically during ...
2
votes
1answer
404 views

Custom error function for randomForest R package

I am a R beginner. Is there a way to specify custom error function with Random Forests in R? For example, say my training data is ,,, so on and my error for any given set needs to normalize the ...
3
votes
2answers
1k views

How to explain poor classification performance of recall when using SVM?

I applied SVM to perform the classification against several data sets. It turns out that the performance metric of recall is pretty bad for one data set. It has recall around 50% while other data sets ...
1
vote
0answers
44 views

Building document exemplar training models for SVM

What are the best methods for building document exemplar training sets for classification of unstructured data (documents and emails) using SVM? How do I optimize F-scores for these models when using ...
2
votes
2answers
204 views

Memory-based learning: Predicting gender of French nouns

I'm trying to predict the gender of French nouns based on their suffixes. I have a corpus of 10k nouns. For each noun, I split the root from the suffix. I create five instances but with varying suffix ...
3
votes
1answer
302 views

Modeling null transitions in the Hidden Markov Model for use with the Viterbi algorithm

I've implemented the classic HMM model from Rabiner's tutorial for gesture recognition and it has worked well. Now, I'm trying to implement the HMM Threshold Model which calls for an HMM with null ...
3
votes
3answers
498 views

How to determine if a price decrease is historically significant?

I am creating a program to collect and analyze prices for goods and alert the user when the price decreases so that they can buy it on "sale." I know a little statistics (e.g. standard deviation) but ...