Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Filter by
Sorted by
Tagged with
108
votes
7answers
15k views

Detecting a given face in a database of facial images

I'm working on a little project involving the faces of twitter users via their profile pictures. A problem I've encountered is that after I filter out all but the images that are clear portrait ...
205
votes
4answers
86k views

ROC vs precision-and-recall curves

I understand the formal differences between them, what I want to know is when it is more relevant to use one vs. the other. Do they always provide complementary insight about the performance of a ...
14
votes
3answers
9k views

GLMNET or LARS for computing LASSO solutions?

I would like to get the coefficients for the LASSO problem $$||Y-X\beta||+\lambda ||\beta||_1.$$ The problem is that glmnet and lars functions give different answers. For the glmnet function I ask ...
11
votes
1answer
1k views

Documented/reproducible examples of successful real-world applications of econometric methods?

This question might sound very broad, but here is what I am looking for. I know there are many excellent books about econometric methods, and many excellent expository articles about econometric ...
6
votes
1answer
463 views

Any good reference books/material to help me build a txn level fraud detection model?

I am looking for a book/case study etc on how to build a fraud detection model at the transaction level. Something applied rather than theoretical would be really helpful.
7
votes
2answers
5k views

cforest and randomForest classification prediction error

I used cforest and randomForest for a 300 rows and 9 columns dataset and received good (almost overfitted - error equal to zero) results for randomForest and big prediction errors for cforest ...
7
votes
1answer
337 views

What are the major machine learning theories that maybe used by Twitter for suggesting followers?

What are the major machine learning theories that maybe used by Twitter for suggesting followers?
10
votes
4answers
2k views

Building MATLAB and R interfaces to Ross Quinlan's C5.0

I'm considering building MATLAB and R interfaces to Ross Quinlan's C5.0 (for those not familiar with it, C5.0 is a decision tree algorithm and software package; an extension of C4.5), and I am trying ...
9
votes
1answer
2k views

Least angle regression keeps the correlations monotonically decreasing and tied?

I'm trying to solve a problem for least angle regression (LAR). This is a problem 3.23 on page 97 of Hastie et al., Elements of Statistical Learning, 2nd. ed. (5th printing). Consider a regression ...
4
votes
2answers
1k views

Understanding expectation maximization for simple 2 linear mixture case

I would appreciate some help getting some EM stuff straight. So, say I generate data in R as follows: ...
21
votes
1answer
5k views

Calibrating a multi-class boosted classifier

I have read Alexandru Niculescu-Mizil and Rich Caruana's paper "Obtaining Calibrated Probabilities from Boosting" and the discussion in this thread. However, I am still having trouble understanding ...
5
votes
2answers
291 views

Ideas for Segmenting Audio Data

I'm currently working on audio data trying to perform a recognition for given classes (for example grinding coffee etc.). However, I have some trouble distinguishing the null class from interesting ...
10
votes
1answer
12k views

Help understand kNN for multi-dimensional data

I understand the premise of kNN algorithm for spatial data. And I know I can extend that algorithm to be used on any continuous data variable (or nominal data with Hamming Distance). However, what ...
6
votes
1answer
967 views

When is the shrinkage applied in Friedman's stochastic gradient boosting machine?

In boosting, each additional tree is fitted to the unexplained variation in the response that is currently un-modelled. If we are using squared-error loss, this amounts to fitting on the residuals ...
20
votes
2answers
2k views

When is “Nearest Neighbor” meaningful, today?

In 1999, Beyer et al. asked, When is "Nearest Neighbor" meaningful? Are there better ways of analyzing and visualizing the effect of distance flatness on NN search since 1999? Does [a given] data ...
28
votes
1answer
6k views

What is behind Google Prediction API?

Google Prediction API is a cloud service where user can submit some training data to train some mysterious classifier and later ask it to classify incoming data, for instance to implement spam filters ...
12
votes
2answers
10k views

Akinator.com and Naive Bayes classifier

Context: I am a programmer with some (half-forgotten) experience in statistics from uni courses. Recently I stumbled upon http://akinator.com and spent some time trying to make it fail. And who wasn't?...
3
votes
2answers
622 views

Worst classifier

What is the worst classier that learns badly in practical problems? Edit: Especially bad on test data.. Thanks
22
votes
2answers
7k views

Cross Validation (error generalization) after model selection

Note: Case is n>>p I am reading Elements of Statistical Learning and there are various mentions about the "right" way to do cross validation( e.g. page 60, page 245). Specifically, my question is how ...
3
votes
1answer
2k views

What is the connection between Kernel Logistic Regression and Smoothing Splines?

Working on probabilistic outputs of kernel methods I found the formulation of the SVM as a Penalized Method using the Binomial Deviance (described for example in "The Elements of Statistical Learning ...
6
votes
1answer
2k views

Leave-one-out cross validation and boosted regression trees

Colleagues of mine recently presented a work where they calibrate boosted regression trees (BRT) models on small data sets ($n= 30$). They validated the models using leave-one-out cross validation (...
7
votes
2answers
4k views

R Package GBM - Bernoulli Deviance

All, I am trying to study the GBM package in R. I. I wanted to try and figure out where the deviance, initial value, gradient and terminal node estimates came from. Please see this snippet: To ...
19
votes
2answers
4k views

Backpropagation algorithm

I got a slight confusion on the backpropagation algorithm used in multilayer perceptron (MLP). The error is adjusted by the cost function. In backpropagation, we are trying to adjust the weight of ...
4
votes
2answers
4k views

Problems with implementing PCA in Matlab

I'm implementing PCA using eigenvalue decomposition in Matlab. I know Matlab has PCA implemented, but it helps me understand all the technicalities when I write code. I've been following the guidance ...
4
votes
2answers
3k views

How to assess overfitting?

This is a follow-up of the question I posted earlier. I am assessing the two RF models which are generated using two different set of features NF - Test_Accuracy > Training accuracy (500 features) ...
5
votes
2answers
1k views

Statistical validation of RandomForest models

I am currently working on a RandomForest based prediction method using protein sequence data. I have generated two models first model (NF) using standard set of features and the second model (HF) ...
20
votes
2answers
19k views

Computing the decision boundary of a linear SVM model

Given the support vectors of a linear SVM, how can I compute the equation of the decision boundary?
218
votes
13answers
200k views

What is the difference between data mining, statistics, machine learning and AI?

What is the difference between data mining, statistics, machine learning and AI? Would it be accurate to say that they are 4 fields attempting to solve very similar problems but with different ...
5
votes
3answers
2k views

Using k-fold cross-validation to test all data

Is it possible to do k-fold cross-validation to test all data, rather than using kfcv to find the optimal hypothesis as is typically done. Example: Say I want to use a svms on a dataset of size 1000....
1
vote
2answers
227 views

How to predict Ozone concentration in few years time?

my friend is a chemist and his problem is to predict the level of ozone concentration in a single site. We have the data for the last 12 years. We want to predict the concentration for the coming ...
5
votes
1answer
300 views

Statistical query model algorithms?

Can you give me examples of machine learning algorithms which learn from the statistical properties of the dataset not the individual observations itself i.e. employ the statistical query model?
2
votes
2answers
357 views

Latest article or new development in cross validation?

Could anybody provide me a latest material related to Cross validation especially R package?
4
votes
1answer
338 views

Factored Joint Distribution of Tree Augmented Naive Bayes Algorithm

I asked this question at Mathoverflow but they recommend me to ask here. I need to find factored joint distribution of Tree Augmented Naive Bayes algorithm. I read the paper but I couldn't figure out ...
8
votes
1answer
298 views

Random generation of scores similar to those of a classification model

Hello fellow number crunchers I want to generate n random scores (together with a class label) as if they had been produced by a binary classification model. In detail, the following properties are ...
2
votes
1answer
1k views

MLE for Naive Bayes in R

I am using the naivebayes function of the e1071 library. Some example commands are: ...
6
votes
1answer
5k views

Measuring statistical significance of machine learning algorithms comparison

Let us consider a comparison of two machine learning algorithms (A and B) on some dataset. Results (root mean squared error) of both algorithms depend on randomly generated initial approximation (...
79
votes
1answer
8k views

Help me understand Support Vector Machines

I understand the basics of what a Support Vector Machines' aim is in terms of classifying an input set into several different classes, but what I don't understand is some of the nitty-gritty details. ...
5
votes
1answer
2k views

Looking for impl of a Fuzzy Neural Network (FNN or SOFNN)

I'm looking for an implementation of FNN (or better yet, a SOFNN as described by Forecasting Time Series by SOFNN with Reinforcement Learning). Any language, though preference is Java, C#, C++ in ...
24
votes
5answers
9k views

Alternatives to classification trees, with better predictive (e.g: CV) performance?

I am looking for an alternative to Classification Trees which might yield better predictive power. The data I am dealing with has factors for both the explanatory and the explained variables. I ...
2
votes
1answer
74 views

Analysis of variables of varying numbers

I work with amino acid sequences and I want to use a self-made model to tell me something about it, lets call it $f(\text{seq})$. Now i want to know the contribution of every position in the sequence ...
14
votes
2answers
4k views

Ordering of time series for machine learning

After reading one of the "Research tips" of R.J. Hyndman about cross-validation and time series, I came back to an old question of mine that I'll try to formulate here. The idea is that in ...
8
votes
3answers
6k views

Clustering genes in a time course experiment

I have seen a few queries on clustering in time series and specifically on clustering, but I don't think they answer my question. Background: I want to cluster genes in a time course experiment in ...
9
votes
1answer
179 views

Learning the Structure of a Hierarchical Reinforcement Task

I've been studying hierachial reinforcement learning problems, and while a lot of papers propose algorithms for learning a policy, they all seem to assume they know in advance a graph structure ...
5
votes
1answer
137 views

Tracking Algorithms

For my thesis I am implementing a tracking algorithm that tracks clusters of datapoints. However, I have a hard time finding research papers and/or an overview of commonly used algoritms in this ...
4
votes
4answers
3k views

PCA on out-of-sample data

We know that the projection matrix learned by PCA can be applied to out-of-sample data points to get their low-dimensional embedding. However, how reliable are these embeddings expected to be, as ...
4
votes
3answers
723 views

What is an “If-Then” rule?

Can someone give a concise, layman's explanation of an "If-Then" rule (as in rule-based systems). I am finding this term used frequently without anyone really defining it properly.
4
votes
2answers
142 views

How good is my error? [closed]

I'm trying to calculate how good are my measurements in machine learning! Let's say that I have five choices, and that error is 4, 2, 0.002, 3, 6. Naturally, I will pick third one for the hit, but I ...
3
votes
3answers
2k views

When does rules based classifier outperforms decision trees?

I am trying to identify approximate 3% of the population for some characteristic feature. Standard decision tree or logistic regression gives too many false positives. Is there a chance that rules ...
15
votes
2answers
7k views

Why does the random forest OOB estimate of error improve when the number of features selected are decreased?

I am applying a random forest algorithm as a classifier on a microarray dataset which are split into two known groups with 1000s of features. After the initial run I look at the importance of the ...
79
votes
3answers
61k views

Best way to present a random forest in a publication?

I am using the random forest algorithm as a robust classifier of two groups in a microarray study with 1000s of features. What is the best way to present the random forest so that there is enough ...