Questions tagged [machine-learning]

Methods and principles of building "computer systems that try to automatically improve with experience."

11,963 questions
1k views

What is the connection between Kernel Logistic Regression and Smoothing Splines?

Working on probabilistic outputs of kernel methods I found the formulation of the SVM as a Penalized Method using the Binomial Deviance (described for example in "The Elements of Statistical Learning ...
1k views

Leave-one-out cross validation and boosted regression trees

Colleagues of mine recently presented a work where they calibrate boosted regression trees (BRT) models on small data sets ($n= 30$). They validated the models using leave-one-out cross validation (...
4k views

R Package GBM - Bernoulli Deviance

All, I am trying to study the GBM package in R. I. I wanted to try and figure out where the deviance, initial value, gradient and terminal node estimates came from. Please see this snippet: To ...
3k views

Backpropagation algorithm

I got a slight confusion on the backpropagation algorithm used in multilayer perceptron (MLP). The error is adjusted by the cost function. In backpropagation, we are trying to adjust the weight of ...
4k views

Problems with implementing PCA in Matlab

I'm implementing PCA using eigenvalue decomposition in Matlab. I know Matlab has PCA implemented, but it helps me understand all the technicalities when I write code. I've been following the guidance ...
2k views

How to assess overfitting?

This is a follow-up of the question I posted earlier. I am assessing the two RF models which are generated using two different set of features NF - Test_Accuracy > Training accuracy (500 ...
1k views

Statistical validation of RandomForest models

I am currently working on a RandomForest based prediction method using protein sequence data. I have generated two models first model (NF) using standard set of features and the second model (HF) ...
14k views

Computing the decision boundary of a linear SVM model

Given the support vectors of a linear SVM, how can I compute the equation of the decision boundary?
186k views

What is the difference between data mining, statistics, machine learning and AI?

What is the difference between data mining, statistics, machine learning and AI? Would it be accurate to say that they are 4 fields attempting to solve very similar problems but with different ...
2k views

Using k-fold cross-validation to test all data

Is it possible to do k-fold cross-validation to test all data, rather than using kfcv to find the optimal hypothesis as is typically done. Example: Say I want to use a svms on a dataset of size 1000....
202 views

How to predict Ozone concentration in few years time?

my friend is a chemist and his problem is to predict the level of ozone concentration in a single site. We have the data for the last 12 years. We want to predict the concentration for the coming ...
258 views

Statistical query model algorithms?

Can you give me examples of machine learning algorithms which learn from the statistical properties of the dataset not the individual observations itself i.e. employ the statistical query model?
353 views

Latest article or new development in cross validation?

Could anybody provide me a latest material related to Cross validation especially R package?
308 views

Factored Joint Distribution of Tree Augmented Naive Bayes Algorithm

I asked this question at Mathoverflow but they recommend me to ask here. I need to find factored joint distribution of Tree Augmented Naive Bayes algorithm. I read the paper but I couldn't figure out ...
263 views

Random generation of scores similar to those of a classification model

Hello fellow number crunchers I want to generate n random scores (together with a class label) as if they had been produced by a binary classification model. In detail, the following properties are ...
1k views

MLE for Naive Bayes in R

I am using the naivebayes function of the e1071 library. Some example commands are: ...
4k views

Measuring statistical significance of machine learning algorithms comparison

Let us consider a comparison of two machine learning algorithms (A and B) on some dataset. Results (root mean squared error) of both algorithms depend on randomly generated initial approximation (...
6k views

Help me understand Support Vector Machines

I understand the basics of what a Support Vector Machines' aim is in terms of classifying an input set into several different classes, but what I don't understand is some of the nitty-gritty details. ...
1k views

Looking for impl of a Fuzzy Neural Network (FNN or SOFNN)

I'm looking for an implementation of FNN (or better yet, a SOFNN as described by Forecasting Time Series by SOFNN with Reinforcement Learning). Any language, though preference is Java, C#, C++ in ...
6k views

Alternatives to classification trees, with better predictive (e.g: CV) performance?

I am looking for an alternative to Classification Trees which might yield better predictive power. The data I am dealing with has factors for both the explanatory and the explained variables. I ...
72 views

Analysis of variables of varying numbers

I work with amino acid sequences and I want to use a self-made model to tell me something about it, lets call it $f(\text{seq})$. Now i want to know the contribution of every position in the sequence ...
3k views

Ordering of time series for machine learning

After reading one of the "Research tips" of R.J. Hyndman about cross-validation and time series, I came back to an old question of mine that I'll try to formulate here. The idea is that in ...
5k views

Clustering genes in a time course experiment

I have seen a few queries on clustering in time series and specifically on clustering, but I don't think they answer my question. Background: I want to cluster genes in a time course experiment in ...
138 views

Learning the Structure of a Hierarchical Reinforcement Task

I've been studying hierachial reinforcement learning problems, and while a lot of papers propose algorithms for learning a policy, they all seem to assume they know in advance a graph structure ...
129 views

Tracking Algorithms

For my thesis I am implementing a tracking algorithm that tracks clusters of datapoints. However, I have a hard time finding research papers and/or an overview of commonly used algoritms in this ...
2k views

PCA on out-of-sample data

We know that the projection matrix learned by PCA can be applied to out-of-sample data points to get their low-dimensional embedding. However, how reliable are these embeddings expected to be, as ...
613 views

What is an “If-Then” rule?

Can someone give a concise, layman's explanation of an "If-Then" rule (as in rule-based systems). I am finding this term used frequently without anyone really defining it properly.
140 views

How good is my error?

I'm trying to calculate how good are my measurements in machine learning! Let's say that I have five choices, and that error is 4, 2, 0.002, 3, 6. Naturally, I will pick third one for the hit, but I ...
2k views

When does rules based classifier outperforms decision trees?

I am trying to identify approximate 3% of the population for some characteristic feature. Standard decision tree or logistic regression gives too many false positives. Is there a chance that rules ...
5k views

Why does the random forest OOB estimate of error improve when the number of features selected are decreased?

I am applying a random forest algorithm as a classifier on a microarray dataset which are split into two known groups with 1000s of features. After the initial run I look at the importance of the ...
55k views

Best way to present a random forest in a publication?

I am using the random forest algorithm as a robust classifier of two groups in a microarray study with 1000s of features. What is the best way to present the random forest so that there is enough ...
26k views

Feature selection for “final” model when performing cross-validation in machine learning

I am getting a bit confused about feature selection and machine learning and I was wondering if you could help me out. I have a microarray dataset that is classified into two groups and has 1000s of ...
7k views

Recommended books or articles as introduction to Cluster Analysis?

I'm working on a small (200M) corpus of text, which I want to explore with some cluster analysis. What books or articles on that subject would you recommend?
41k views

What's the difference between feed-forward and recurrent neural networks?

What is the difference between a feed-forward and recurrent neural network? Why would you use one over the other? Do other network topologies exist?
17k views

Variable importance from SVM

How to obtain a variable (attribute) importance using SVM?
2k views

Applying the “kernel trick” to linear methods?

The kernel trick is used in several machine learning models (e.g. SVM). It was first introduced in the "Theoretical foundations of the potential function method in pattern recognition learning" paper ...
143 views

For data similar to audio, how to determine if there are 1 or 2 categories? [closed]

I have to compare pairs of audio strems as 1d time series. Looking at the aligned trajectories I need to either cluster them together or assume they arise from independent generators. I remember ...
1k views

Machine Learning conferences? [closed]

What are the most significant annual Machine Learning conferences? Rules: One conference per answer Include a link to the conference
4k views

Application of machine learning techniques in small sample clinical studies

What do you think about applying machine learning techniques, like Random Forests or penalized regression (with L1 or L2 penalty, or a combination thereof) in small sample clinical studies when the ...
1k views

Few machine learning problems

In a particular application I was in need of machine learning (I know the things I studied in my undergraduate course). I used Support Vector Machines and got the problem solved. Its working fine. ...
30k views

What is the weak side of decision trees?

Decision trees seems to be a very understandable machine learning method. Once created it can be easily inspected by a human which is a great advantage in some applications. What are the practical ...
5k views

SVD dimensionality reduction for time series of different length

I am using Singular Value Decomposition as a dimensionality reduction technique. Given N vectors of dimension D, the idea is to ...
14k views

Free data set for very high dimensional classification [closed]

What are the freely available data set for classification with more than 1000 features (or sample points if it contains curves)? There is already a community wiki about free data sets: Locating ...
28k views

Online vs offline learning?

What is the difference between offline and online learning? Is it just a matter of learning over the entire dataset (offline) vs. learning incrementally (one instance at a time)? What are examples ...
14k views

What is the 'fundamental' idea of machine learning for estimating parameters?

The 'fundamental' idea of statistics for estimating parameters is maximum likelihood. I am wondering what is the corresponding idea in machine learning. Qn 1. Would it be fair to say that the '...
721 views

Cross validation in very high dimension (to select the number of used variables in very high dimensional classification)

My question is about cross validation when there are many more variables than observations. To fix ideas, I propose to restrict to the classification framework in very high dimension (more features ...
325 views

Is it possible to use machine learning as a method for learning stats, rather than vice-versa?

During every machine learning tutorial you'll find, there is the common "You will need to know x amount of stats before starting this tutorial". As such, using your knowledge of stats, you will learn ...
1k views

Suggested R packages for frontier estimation or segmentation of hyperspectral images

An hyperspectral image is a multidimensional image with more than 200 spectral bands i.e. an image for which each pixel is a vector of dimension 200 (most often it is a sampled spectral curve that is ...