Questions tagged [machine-learning]
Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.
16,632
questions
108
votes
7answers
15k views
Detecting a given face in a database of facial images
I'm working on a little project involving the faces of twitter users via their profile pictures.
A problem I've encountered is that after I filter out all but the images that are clear portrait ...
205
votes
4answers
86k views
ROC vs precision-and-recall curves
I understand the formal differences between them, what I want to know is when it is more relevant to use one vs. the other.
Do they always provide complementary insight about the performance of a ...
14
votes
3answers
9k views
GLMNET or LARS for computing LASSO solutions?
I would like to get the coefficients for the LASSO problem
$$||Y-X\beta||+\lambda ||\beta||_1.$$
The problem is that glmnet and lars functions give different answers. For the glmnet function I ask ...
11
votes
1answer
1k views
Documented/reproducible examples of successful real-world applications of econometric methods?
This question might sound very broad, but here is what I am looking for. I know there are many excellent books about econometric methods, and many excellent expository articles about econometric ...
6
votes
1answer
463 views
Any good reference books/material to help me build a txn level fraud detection model?
I am looking for a book/case study etc on how to build a fraud detection model at the transaction level. Something applied rather than theoretical would be really helpful.
7
votes
2answers
5k views
cforest and randomForest classification prediction error
I used cforest and randomForest for a 300 rows and 9 columns dataset and received good (almost overfitted - error equal to zero) results for randomForest and big prediction errors for cforest ...
7
votes
1answer
337 views
What are the major machine learning theories that maybe used by Twitter for suggesting followers?
What are the major machine learning theories that maybe used by Twitter for suggesting followers?
10
votes
4answers
2k views
Building MATLAB and R interfaces to Ross Quinlan's C5.0
I'm considering building MATLAB and R interfaces to Ross Quinlan's C5.0 (for those not familiar with it, C5.0 is a decision tree algorithm and software package; an extension of C4.5), and I am trying ...
9
votes
1answer
2k views
Least angle regression keeps the correlations monotonically decreasing and tied?
I'm trying to solve a problem for least angle regression (LAR). This is a problem 3.23 on page 97 of Hastie et al., Elements of Statistical Learning, 2nd. ed. (5th printing).
Consider a regression ...
4
votes
2answers
1k views
Understanding expectation maximization for simple 2 linear mixture case
I would appreciate some help getting some EM stuff straight. So, say I generate data in R as follows:
...
21
votes
1answer
5k views
Calibrating a multi-class boosted classifier
I have read Alexandru Niculescu-Mizil and Rich Caruana's paper "Obtaining Calibrated Probabilities from Boosting" and the discussion in this thread. However, I am still having trouble understanding ...
5
votes
2answers
291 views
Ideas for Segmenting Audio Data
I'm currently working on audio data trying to perform a recognition for given classes (for example grinding coffee etc.). However, I have some trouble distinguishing the null class from interesting ...
10
votes
1answer
12k views
Help understand kNN for multi-dimensional data
I understand the premise of kNN algorithm for spatial data. And I know I can extend that algorithm to be used on any continuous data variable (or nominal data with Hamming Distance). However, what ...
6
votes
1answer
967 views
When is the shrinkage applied in Friedman's stochastic gradient boosting machine?
In boosting, each additional tree is fitted to the unexplained variation in the response that is currently un-modelled. If we are using squared-error loss, this amounts to fitting on the residuals ...
20
votes
2answers
2k views
When is “Nearest Neighbor” meaningful, today?
In 1999, Beyer et al. asked,
When is "Nearest Neighbor" meaningful?
Are there better ways of analyzing and visualizing
the effect of distance flatness on NN search since 1999?
Does [a given] data ...
28
votes
1answer
6k views
What is behind Google Prediction API?
Google Prediction API is a cloud service where user can submit some training data to train some mysterious classifier and later ask it to classify incoming data, for instance to implement spam filters ...
12
votes
2answers
10k views
Akinator.com and Naive Bayes classifier
Context: I am a programmer with some (half-forgotten) experience in statistics from uni courses. Recently I stumbled upon http://akinator.com and spent some time trying to make it fail. And who wasn't?...
3
votes
2answers
622 views
Worst classifier
What is the worst classier that learns badly in practical problems?
Edit:
Especially bad on test data..
Thanks
22
votes
2answers
7k views
Cross Validation (error generalization) after model selection
Note: Case is n>>p
I am reading Elements of Statistical Learning and there are various mentions about the "right" way to do cross validation( e.g. page 60, page 245). Specifically, my question is how ...
3
votes
1answer
2k views
What is the connection between Kernel Logistic Regression and Smoothing Splines?
Working on probabilistic outputs of kernel methods I found the formulation of the SVM as a Penalized Method using the Binomial Deviance (described for example in "The Elements of Statistical Learning ...
6
votes
1answer
2k views
Leave-one-out cross validation and boosted regression trees
Colleagues of mine recently presented a work where they calibrate boosted regression trees (BRT) models on small data sets ($n= 30$). They validated the models using leave-one-out cross validation (...
7
votes
2answers
4k views
R Package GBM - Bernoulli Deviance
All,
I am trying to study the GBM package in R.
I. I wanted to try and figure out where the deviance, initial value, gradient and terminal node estimates came from. Please see this snippet:
To ...
19
votes
2answers
4k views
Backpropagation algorithm
I got a slight confusion on the backpropagation algorithm used in multilayer perceptron (MLP).
The error is adjusted by the cost function. In backpropagation, we are trying to adjust the weight of ...
4
votes
2answers
4k views
Problems with implementing PCA in Matlab
I'm implementing PCA using eigenvalue decomposition in Matlab. I know Matlab has PCA implemented, but it helps me understand all the technicalities when I write code.
I've been following the guidance ...
4
votes
2answers
3k views
How to assess overfitting?
This is a follow-up of the question I posted earlier.
I am assessing the two RF models which are generated using two different set of features
NF - Test_Accuracy > Training accuracy (500 features)
...
5
votes
2answers
1k views
Statistical validation of RandomForest models
I am currently working on a RandomForest based prediction method using protein sequence data. I have generated two models first model (NF) using standard set of features and the second model (HF) ...
20
votes
2answers
19k views
Computing the decision boundary of a linear SVM model
Given the support vectors of a linear SVM, how can I compute the equation of the decision boundary?
218
votes
13answers
200k views
What is the difference between data mining, statistics, machine learning and AI?
What is the difference between data mining, statistics, machine learning and AI?
Would it be accurate to say that they are 4 fields attempting to solve very similar problems but with different ...
5
votes
3answers
2k views
Using k-fold cross-validation to test all data
Is it possible to do k-fold cross-validation to test all data, rather than using kfcv to find the optimal hypothesis as is typically done.
Example:
Say I want to use a svms on a dataset of size 1000....
1
vote
2answers
227 views
How to predict Ozone concentration in few years time?
my friend is a chemist and his problem is to predict the level of ozone concentration in a single site. We have the data for the last 12 years.
We want to predict the concentration for the coming ...
5
votes
1answer
300 views
Statistical query model algorithms?
Can you give me examples of machine learning algorithms which learn from the statistical properties of the dataset not the individual observations itself i.e. employ the statistical query model?
2
votes
2answers
357 views
Latest article or new development in cross validation?
Could anybody provide me a latest material related to Cross validation especially R package?
4
votes
1answer
338 views
Factored Joint Distribution of Tree Augmented Naive Bayes Algorithm
I asked this question at Mathoverflow but they recommend me to ask here. I need to find factored joint distribution of Tree Augmented Naive Bayes algorithm. I read the paper but I couldn't figure out ...
8
votes
1answer
298 views
Random generation of scores similar to those of a classification model
Hello fellow number crunchers
I want to generate n random scores (together with a class label) as if they had been produced by a binary classification model. In detail, the following properties are ...
2
votes
1answer
1k views
MLE for Naive Bayes in R
I am using the naivebayes function of the e1071 library. Some example commands are:
...
6
votes
1answer
5k views
Measuring statistical significance of machine learning algorithms comparison
Let us consider a comparison of two machine learning algorithms (A and B) on some dataset. Results (root mean squared error) of both algorithms depend on randomly generated initial approximation (...
79
votes
1answer
8k views
Help me understand Support Vector Machines
I understand the basics of what a Support Vector Machines' aim is in terms of classifying an input set into several different classes, but what I don't understand is some of the nitty-gritty details. ...
5
votes
1answer
2k views
Looking for impl of a Fuzzy Neural Network (FNN or SOFNN)
I'm looking for an implementation of FNN (or better yet, a SOFNN as described by Forecasting Time Series by SOFNN with Reinforcement Learning). Any language, though preference is Java, C#, C++ in ...
24
votes
5answers
9k views
Alternatives to classification trees, with better predictive (e.g: CV) performance?
I am looking for an alternative to Classification Trees which might yield better predictive power.
The data I am dealing with has factors for both the explanatory and the explained variables.
I ...
2
votes
1answer
74 views
Analysis of variables of varying numbers
I work with amino acid sequences and I want to use a self-made model to tell me something about it, lets call it $f(\text{seq})$. Now i want to know the contribution of every position in the sequence ...
14
votes
2answers
4k views
Ordering of time series for machine learning
After reading one of the "Research tips" of R.J. Hyndman about cross-validation and time series, I came back to an old question of mine that I'll try to formulate here. The idea is that in ...
8
votes
3answers
6k views
Clustering genes in a time course experiment
I have seen a few queries on clustering in time series and specifically on clustering, but I don't think they answer my question.
Background: I want to cluster genes in a time course experiment in ...
9
votes
1answer
179 views
Learning the Structure of a Hierarchical Reinforcement Task
I've been studying hierachial reinforcement learning problems, and while a lot of papers propose algorithms for learning a policy, they all seem to assume they know in advance a graph structure ...
5
votes
1answer
137 views
Tracking Algorithms
For my thesis I am implementing a tracking algorithm that tracks clusters of datapoints.
However, I have a hard time finding research papers and/or an overview of commonly used algoritms in this ...
4
votes
4answers
3k views
PCA on out-of-sample data
We know that the projection matrix learned by PCA can be applied to out-of-sample data points to get their low-dimensional embedding. However, how reliable are these embeddings expected to be, as ...
4
votes
3answers
723 views
What is an “If-Then” rule?
Can someone give a concise, layman's explanation of an "If-Then" rule (as in rule-based systems). I am finding this term used frequently without anyone really defining it properly.
4
votes
2answers
142 views
How good is my error? [closed]
I'm trying to calculate how good are my measurements in machine learning! Let's say that I have five choices, and that error is 4, 2, 0.002, 3, 6. Naturally, I will pick third one for the hit, but I ...
3
votes
3answers
2k views
When does rules based classifier outperforms decision trees?
I am trying to identify approximate 3% of the population for some characteristic feature. Standard decision tree or logistic regression gives too many false positives. Is there a chance that rules ...
15
votes
2answers
7k views
Why does the random forest OOB estimate of error improve when the number of features selected are decreased?
I am applying a random forest algorithm as a classifier on a microarray dataset which are split into two known groups with 1000s of features. After the initial run I look at the importance of the ...
79
votes
3answers
61k views
Best way to present a random forest in a publication?
I am using the random forest algorithm as a robust classifier of two groups in a microarray study with 1000s of features.
What is the best way to present the random forest so that there is enough ...
