Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

Filter by
Sorted by
Tagged with
5 votes
2 answers
1k views

Understanding expectation maximization for simple 2 linear mixture case

I would appreciate some help getting some EM stuff straight. So, say I generate data in R as follows: ...
user avatar
  • 393
23 votes
1 answer
6k views

Calibrating a multi-class boosted classifier

I have read Alexandru Niculescu-Mizil and Rich Caruana's paper "Obtaining Calibrated Probabilities from Boosting" and the discussion in this thread. However, I am still having trouble understanding ...
user avatar
5 votes
2 answers
293 views

Ideas for Segmenting Audio Data

I'm currently working on audio data trying to perform a recognition for given classes (for example grinding coffee etc.). However, I have some trouble distinguishing the null class from interesting ...
user avatar
  • 206
12 votes
1 answer
15k views

Help understand kNN for multi-dimensional data

I understand the premise of kNN algorithm for spatial data. And I know I can extend that algorithm to be used on any continuous data variable (or nominal data with Hamming Distance). However, what ...
user avatar
6 votes
1 answer
1k views

When is the shrinkage applied in Friedman's stochastic gradient boosting machine?

In boosting, each additional tree is fitted to the unexplained variation in the response that is currently un-modelled. If we are using squared-error loss, this amounts to fitting on the residuals ...
user avatar
22 votes
2 answers
2k views

When is "Nearest Neighbor" meaningful, today?

In 1999, Beyer et al. asked, When is "Nearest Neighbor" meaningful? Are there better ways of analyzing and visualizing the effect of distance flatness on NN search since 1999? Does [a given] data ...
user avatar
  • 3,207
28 votes
1 answer
6k views

What is behind Google Prediction API?

Google Prediction API is a cloud service where user can submit some training data to train some mysterious classifier and later ask it to classify incoming data, for instance to implement spam filters ...
user avatar
12 votes
2 answers
10k views

Akinator.com and Naive Bayes classifier

Context: I am a programmer with some (half-forgotten) experience in statistics from uni courses. Recently I stumbled upon http://akinator.com and spent some time trying to make it fail. And who wasn't?...
user avatar
  • 221
3 votes
2 answers
690 views

Worst classifier

What is the worst classier that learns badly in practical problems? Edit: Especially bad on test data.. Thanks
user avatar
  • 273
24 votes
2 answers
8k views

Cross Validation (error generalization) after model selection

Note: Case is n>>p I am reading Elements of Statistical Learning and there are various mentions about the "right" way to do cross validation( e.g. page 60, page 245). Specifically, my question is how ...
user avatar
  • 7,660
4 votes
1 answer
2k views

What is the connection between Kernel Logistic Regression and Smoothing Splines?

Working on probabilistic outputs of kernel methods I found the formulation of the SVM as a Penalized Method using the Binomial Deviance (described for example in "The Elements of Statistical Learning ...
user avatar
  • 391
6 votes
1 answer
2k views

Leave-one-out cross validation and boosted regression trees

Colleagues of mine recently presented a work where they calibrate boosted regression trees (BRT) models on small data sets ($n= 30$). They validated the models using leave-one-out cross validation (...
user avatar
7 votes
2 answers
4k views

R Package GBM - Bernoulli Deviance

All, I am trying to study the GBM package in R. I. I wanted to try and figure out where the deviance, initial value, gradient and terminal node estimates came from. Please see this snippet: To ...
user avatar
  • 7,660
22 votes
3 answers
6k views

Backpropagation algorithm and error in hidden layer

I got a slight confusion on the backpropagation algorithm used in multilayer perceptron (MLP). The error is adjusted by the cost function. In backpropagation, we are trying to adjust the weight of ...
user avatar
  • 489
4 votes
2 answers
4k views

Problems with implementing PCA in Matlab

I'm implementing PCA using eigenvalue decomposition in Matlab. I know Matlab has PCA implemented, but it helps me understand all the technicalities when I write code. I've been following the guidance ...
user avatar
  • 375
4 votes
2 answers
3k views

How to assess overfitting?

This is a follow-up of the question I posted earlier. I am assessing the two RF models which are generated using two different set of features NF - Test_Accuracy > Training accuracy (500 features) ...
user avatar
6 votes
2 answers
1k views

Statistical validation of RandomForest models

I am currently working on a RandomForest based prediction method using protein sequence data. I have generated two models first model (NF) using standard set of features and the second model (HF) ...
user avatar
21 votes
2 answers
21k views

Computing the decision boundary of a linear SVM model

Given the support vectors of a linear SVM, how can I compute the equation of the decision boundary?
user avatar
  • 753
220 votes
13 answers
201k views

What is the difference between data mining, statistics, machine learning and AI?

What is the difference between data mining, statistics, machine learning and AI? Would it be accurate to say that they are 4 fields attempting to solve very similar problems but with different ...
6 votes
3 answers
2k views

Using k-fold cross-validation to test all data

Is it possible to do k-fold cross-validation to test all data, rather than using kfcv to find the optimal hypothesis as is typically done. Example: Say I want to use a svms on a dataset of size 1000....
user avatar
  • 537
1 vote
2 answers
234 views

How to predict Ozone concentration in few years time?

my friend is a chemist and his problem is to predict the level of ozone concentration in a single site. We have the data for the last 12 years. We want to predict the concentration for the coming ...
user avatar
5 votes
1 answer
314 views

Statistical query model algorithms?

Can you give me examples of machine learning algorithms which learn from the statistical properties of the dataset not the individual observations itself i.e. employ the statistical query model?
user avatar
  • 159
2 votes
2 answers
359 views

Latest article or new development in cross validation?

Could anybody provide me a latest material related to Cross validation especially R package?
user avatar
4 votes
1 answer
346 views

Factored Joint Distribution of Tree Augmented Naive Bayes Algorithm

I asked this question at Mathoverflow but they recommend me to ask here. I need to find factored joint distribution of Tree Augmented Naive Bayes algorithm. I read the paper but I couldn't figure out ...
user avatar
8 votes
1 answer
458 views

Random generation of scores similar to those of a classification model

Hello fellow number crunchers I want to generate n random scores (together with a class label) as if they had been produced by a binary classification model. In detail, the following properties are ...
user avatar
  • 10.1k
2 votes
1 answer
1k views

MLE for Naive Bayes in R

I am using the naivebayes function of the e1071 library. Some example commands are: ...
user avatar
7 votes
1 answer
5k views

Measuring statistical significance of machine learning algorithms comparison

Let us consider a comparison of two machine learning algorithms (A and B) on some dataset. Results (root mean squared error) of both algorithms depend on randomly generated initial approximation (...
user avatar
  • 415
82 votes
1 answer
8k views

Help me understand Support Vector Machines

I understand the basics of what a Support Vector Machines' aim is in terms of classifying an input set into several different classes, but what I don't understand is some of the nitty-gritty details. ...
user avatar
  • 1,187
5 votes
1 answer
2k views

Looking for impl of a Fuzzy Neural Network (FNN or SOFNN)

I'm looking for an implementation of FNN (or better yet, a SOFNN as described by Forecasting Time Series by SOFNN with Reinforcement Learning). Any language, though preference is Java, C#, C++ in ...
user avatar
25 votes
5 answers
11k views

Alternatives to classification trees, with better predictive (e.g: CV) performance?

I am looking for an alternative to Classification Trees which might yield better predictive power. The data I am dealing with has factors for both the explanatory and the explained variables. I ...
user avatar
  • 20.1k
2 votes
1 answer
77 views

Analysis of variables of varying numbers

I work with amino acid sequences and I want to use a self-made model to tell me something about it, lets call it $f(\text{seq})$. Now i want to know the contribution of every position in the sequence ...
user avatar
  • 153
14 votes
2 answers
4k views

Ordering of time series for machine learning

After reading one of the "Research tips" of R.J. Hyndman about cross-validation and time series, I came back to an old question of mine that I'll try to formulate here. The idea is that in ...
user avatar
  • 215
8 votes
3 answers
6k views

Clustering genes in a time course experiment

I have seen a few queries on clustering in time series and specifically on clustering, but I don't think they answer my question. Background: I want to cluster genes in a time course experiment in ...
user avatar
  • 6,212
10 votes
1 answer
191 views

Learning the Structure of a Hierarchical Reinforcement Task

I've been studying hierachial reinforcement learning problems, and while a lot of papers propose algorithms for learning a policy, they all seem to assume they know in advance a graph structure ...
user avatar
  • 644
5 votes
1 answer
139 views

Tracking Algorithms

For my thesis I am implementing a tracking algorithm that tracks clusters of datapoints. However, I have a hard time finding research papers and/or an overview of commonly used algoritms in this ...
user avatar
  • 255
5 votes
4 answers
3k views

PCA on out-of-sample data

We know that the projection matrix learned by PCA can be applied to out-of-sample data points to get their low-dimensional embedding. However, how reliable are these embeddings expected to be, as ...
user avatar
  • 2,143
4 votes
3 answers
1k views

What is an "If-Then" rule?

Can someone give a concise, layman's explanation of an "If-Then" rule (as in rule-based systems). I am finding this term used frequently without anyone really defining it properly.
user avatar
  • 265
4 votes
2 answers
143 views

How good is my error? [closed]

I'm trying to calculate how good are my measurements in machine learning! Let's say that I have five choices, and that error is 4, 2, 0.002, 3, 6. Naturally, I will pick third one for the hit, but I ...
user avatar
  • 181
3 votes
3 answers
2k views

When does rules based classifier outperforms decision trees?

I am trying to identify approximate 3% of the population for some characteristic feature. Standard decision tree or logistic regression gives too many false positives. Is there a chance that rules ...
user avatar
15 votes
2 answers
8k views

Why does the random forest OOB estimate of error improve when the number of features selected are decreased?

I am applying a random forest algorithm as a classifier on a microarray dataset which are split into two known groups with 1000s of features. After the initial run I look at the importance of the ...
user avatar
80 votes
3 answers
64k views

Best way to present a random forest in a publication?

I am using the random forest algorithm as a robust classifier of two groups in a microarray study with 1000s of features. What is the best way to present the random forest so that there is enough ...
user avatar
91 votes
6 answers
45k views

Feature selection for "final" model when performing cross-validation in machine learning

I am getting a bit confused about feature selection and machine learning and I was wondering if you could help me out. I have a microarray dataset that is classified into two groups and has 1000s of ...
user avatar
13 votes
6 answers
9k views

Recommended books or articles as introduction to Cluster Analysis?

I'm working on a small (200M) corpus of text, which I want to explore with some cluster analysis. What books or articles on that subject would you recommend?
71 votes
3 answers
94k views

What's the difference between feed-forward and recurrent neural networks?

What is the difference between a feed-forward and recurrent neural network? Why would you use one over the other? Do other network topologies exist?
user avatar
  • 12k
37 votes
3 answers
30k views

Variable importance from SVM

How to obtain a variable (attribute) importance using SVM?
user avatar
20 votes
3 answers
4k views

Applying the "kernel trick" to linear methods?

The kernel trick is used in several machine learning models (e.g. SVM). It was first introduced in the "Theoretical foundations of the potential function method in pattern recognition learning" paper ...
user avatar
  • 12k
1 vote
1 answer
144 views

For data similar to audio, how to determine if there are 1 or 2 categories? [closed]

I have to compare pairs of audio strems as 1d time series. Looking at the aligned trajectories I need to either cluster them together or assume they arise from independent generators. I remember ...
user avatar
  • 1,455
6 votes
12 answers
2k views

Machine Learning conferences? [closed]

What are the most significant annual Machine Learning conferences? Rules: One conference per answer Include a link to the conference
16 votes
5 answers
5k views

Application of machine learning techniques in small sample clinical studies

What do you think about applying machine learning techniques, like Random Forests or penalized regression (with L1 or L2 penalty, or a combination thereof) in small sample clinical studies when the ...
user avatar
  • 51.3k
6 votes
2 answers
1k views

Few machine learning problems

In a particular application I was in need of machine learning (I know the things I studied in my undergraduate course). I used Support Vector Machines and got the problem solved. Its working fine. ...
user avatar
  • 12.7k