Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms (1)

8
votes
0answers
125 views

In statistical learning theory, isn't there a problem of overfitting on a test set?

Let's consider the problem about classifying the MNIST dataset. According to Yann LeCun's MNIST Webpage, 'Ciresan et al.' got 0.23% error rate on MNIST test set using Convolutional Neural Network. ...
8
votes
0answers
379 views

Link Anomaly Detection in Temporal Network

I came across this paper that uses link anomaly detection to predict trending topics, and I found it incredibly intriguing: The paper is "Discovering Emerging Topics in Social Streams via Link Anomaly ...
6
votes
0answers
189 views

Under which conditions do gradient boosting machines outperform random forests?

Can Friedman's gradient boosting machine achieve better performance than random forests? If so, in which conditions or what kind of data set can make gbm better?
6
votes
0answers
902 views

Anomaly Detection with Dummy Features (and other Discrete/Categorical Features)

Intro This is my first time posting on here, so please, if anything doesn't seem technically correct, either in the formatting, or the use of correct definitions, I'm interested to know what ...
6
votes
0answers
95 views

Open-sourced pairwise learning models

I am solving classification problem using pairwise-learning training set. We have 2 classes: bad and good. We also have pairs of objects $(a_i,b_i)_{i=1}^n$, meaning that object $a_i$ is better than ...
6
votes
0answers
930 views

Maximum entropy classifier and sentiment analysis

I am doing a project work in sentiment analysis (on Twitter data) using machine learning approach. In order to find the 'best' way to this I have experimented with naive Bayesian and maximum entropy ...
6
votes
0answers
177 views

Self-organizing maps: fuzzy input?

as my first post I would like to know if there are SOM implementations (preferably R) available that accept fuzzy input. That is, I have data in which some nominal features are spread out between a ...
5
votes
0answers
110 views

Are all models useless? Is any exact model possible — or useful?

This question has been festering in my mind for over a month. The February 2015 issue of Amstat News contains an article by Berkeley Professor Mark van der Laan that scolds people for using inexact ...
5
votes
0answers
120 views

Rademacher complexity of logistic regression

Consider logistic regression. We have the logistic loss function, $\phi: R\rightarrow [0,1], \phi(u)=\log(1+\exp(-u))$, which is Lipschitz, and we have the linear function class $F=\{f_w:R^d ...
5
votes
0answers
141 views

Convolutional neural networks: Aren't the central neurons over-represented in the output?

[This question was also posed at stack overflow] The question in short I'm studying convolutional neural networks, and I believe that these networks do not treat every input neuron ...
5
votes
0answers
220 views

Generalized linear model with lasso regularization for continuous non-negative response

I have a big data problem with a large number of predictors and a non-negative response (time until inspection). For a full model I would use a glm with Gamma distributed response (link="log"). ...
5
votes
0answers
106 views

Expectation Maximization Clarification

I found very helpful tutorial regarding EM algorithm. The example and the picture from the tutorial is simply brilliant. Related question about calculating probabilities how does expectation ...
5
votes
0answers
1k views

Confusion with Vowpal Wabbit's multiple-pass behavior when performing ridge-regression

I have encountered many peculiarities/misunderstandings of Vowpal Wabbit when trying to do online multiple-pass learning. Specifically, I need to solve a Ridge Linear regression problem, with ...
5
votes
0answers
111 views

Compressed sensing: Optimization in $L_1$ norm and total variation with fourier coefficients

I'm reading the article Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information (Candes, Romberg and Tao, 2004). In this article they are talking ...
5
votes
0answers
516 views

Manifold regularization using laplacian graph in SVM

I'm trying implement Manifold Regularization in Support Vector Machines (SVMs) in Matlab. I'm following the instructions in the paper by Belkin et al.(2006), there's the equation in it: $f^{*} = ...
5
votes
0answers
577 views

Post processing random forests using regularised regression: what about bias?

I have been playing around with post processing the results of the random forest for regression machine learning algorithm in order to try and do better than the default mean of all trees prediction. ...
4
votes
0answers
42 views

SVM with non-negative weights

An SVM classifier can be obtained by solving the following, $\arg\min \frac{1}{2}\|W\|_2^2 + C\sum_i \max(0, 1-y_i (W^T\mathbf{x}_i + b))$ where $W$ is the hyperplane (or weights), $b$ is the bias, ...
4
votes
0answers
93 views

Ideal statistical or machine learning technique to model highly cross-correlated data

I'm trying to build a model that can predict streamflow for an alpine (snowmelt-fed) watershed using snow albedo (roughly, the energy reflectance of the snow) data. Albedo controls the melt of the ...
4
votes
0answers
54 views

VC dimension of regression models

In the lecture series Learning from Data, the professor mentions that the VC dimension measures the model complexity on how many points a given model can shatter. So this works perfectly well for ...
4
votes
0answers
126 views

State of the Art versions of Generalized Additive Models

Generalized Additive Models [Tribshirani 86] was well received with over 1335 Citations. I am also aware of the popular version of GAM - the Multivariate Adaptive Regression Splines [MARS by Friedman ...
4
votes
0answers
115 views

Explanation for large difference in SVM and Naive bayes results

I have a dataset with 389 data evenly distributed into 6 classes. Each data has 1024 features. So my dimension is much larger than my observation data. I have tried to see some common classifiers on ...
4
votes
0answers
123 views

Model selection in offline vs. online learning

I've been trying to learn more about online learning lately (it's absolutely fascinating!), and one theme that I haven't been able to get a good grasp on is how to think about model selection in ...
4
votes
0answers
66 views

Vapniks proof of the basic lemma

In his book Statistical Learning Theory (1998), Vladimir Vapnik proves an inequality needed to prove a bound on the risk for indicator loss functions. Theorem 4.1 on page 133 he derives the following ...
4
votes
0answers
342 views

How to perform hypothesis testing for comparing different classifiers

I am trying to classify a small dataset (around 500 records) into two classes. I used various methods like SVM, Naive Bayes and k-nn classifier. Now, I would like to set the results from one of the ...
4
votes
0answers
2k views

Is R-squared value appropriate for comparing models?

I'm trying to identify the best model to predict the prices of automobiles, using the prices and features available on automobile classified advertisement sites. For this I used couple a of models ...
4
votes
0answers
121 views

What kind of plot am I looking at?

I stumbled on to these following two slides (slides 21 & 22 on a machine learning tutorial found here): The first is obviously an $x,y$ scatterplot of height and weight. But what is the ...
4
votes
0answers
66 views

How to model the distribution of a word game in order to find correlation between demographics and chosen words

I have an experiment (in the form of a word game) whereby people are asked to choose a set of words to describe associations with a topic with the aim of having another person guess the topic. I ...
4
votes
0answers
235 views

detecting circadian rhythm in a time series

I have a sensor that can detect minute changes in distance. It produces a time series. I would like to point it at people and detect things like their sleeping pattern. How would one build a system ...
4
votes
0answers
332 views

Incremental learning methods in R

I am looking for some libraries in R that can do incremental learning (also called online or sequential learning). The use case of such learning in comparison to traditional batch methods would be to ...
4
votes
0answers
77 views

How to learn similarity of typed/attributed graphs?

I have a question for graph machine learning gurus :). For this project I'm working on, I need to be able to learn similarity between typed graphs. By typed I mean that every vertex and every edge of ...
4
votes
0answers
99 views

Graphical nominal model

Suppose I have a set of $k$ matrices. $$ \mathbb D = A_1,A_2,...,A_k $$ Each column of $A$ is categorical vector. $$ A = v_1,v_2,...,v_n $$ I want to find the mapping $$ f: A ...
4
votes
0answers
190 views

Category selection for text classification

It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
3
votes
0answers
60 views

How to build “supervised clustering” for neural networks?

I'm confused as to what the output would be. Consider the "blind source separation" problem. Let's say I have a ton of training examples where the input is the final cacophony of sounds as a sound ...
3
votes
0answers
11 views

Inter-rater agreement of a gold standard dataset - a ceiling for reliable evaluation of algorithms?

In my field, a dated gold standard dataset is used to track progress in algorithm development. Now when the state-of-the-art algorithms obtain higher correlation than is the inter-rater agreement of ...
3
votes
0answers
39 views

Concept of fitting cubic splines

i am reading the elements of statistical learning, and the following diagram is included on page 262. i interpret the text (potentially incorrectly) that the fitted line $\mu(x)$ is a linear ...
3
votes
0answers
77 views

Should I make decisions based on micro-averaged or macro-averaged evaluation measures?

I ran a 10-fold cross validation on different binary classification algortihms, with the same dataset, and recieved both Micro- and Macro averaged results. It should be mentioned that this was a ...
3
votes
0answers
41 views

Drunken dancer. Best fit of an experimental trajectory to model trajectories with large variations in independent variable

I have several model point trajectories (choreographies), describing movement of a model dancer in a one (for simplicity) dimensional space versus time. I am also observing a movement of a real ...
3
votes
0answers
57 views

Markov decision process in R for a song suggestion software?

We have a music player that has different playlists and automatically suggests songs from the current playlist I'm in. What I want the program to learn is, that if I skip the song, it should decrease ...
3
votes
0answers
38 views

Variance of binomial vs. multinomial distribution in cross-validation

Suppose we have a dataset with $N=100$ observations. We do $K$-fold cross-validation with $K=10$ and $K=100$. In the first case, the classification decisions are sampled (can I say it like this?) ...
3
votes
0answers
26 views

Bound on the total change using Pearson's r

I am given an increasing series $(x_1,....x_n)$ and I know the pearson correlation between $(x_1,....x_n)$ and some (unknown) increasing series $(y_1,....y_n)$. Can I derive an upper and a lower ...
3
votes
0answers
33 views

Heuristics streaming data matching

I have an index composed by thousands of documents. Slightly modified copies of those documents are sent to my application in small chunks, and I need to check, from those chunks, which document has ...
3
votes
0answers
50 views

When to use Bayesian Networks over other machine learning approaches?

I expect there may be no definitive answer to this question. But I have used a number of machine learning algorithms in the past and am trying to learn about Bayesian Networks. I would like to ...
3
votes
0answers
49 views

Is it possible for a reinforcement learning agent to create or generate additional features

I have little background knowledge of Machine Learning, so please forgive me if my question seems silly. Based on what I've read, the best model-free reinforcement learning algorithm to this date is ...
3
votes
0answers
57 views

Bounding the expectation of the difference between empirical vs generalization error

Let the (defect) difference between empirical and generalization error be: $$D[f_S] = I_S[f_S] - I[f_S]$$ where the empirical risk is: $$I_S[f_S] = \frac{1}{n}\sum^n_{i=1} V(f_S,z_i)$$ where ...
3
votes
0answers
143 views

What enforces features diversity in RBM?

I'm working on an implementation of a Restricted Boltzman Machine (RBM). I made some tests on the MNIST dataset trying to learn a representation of the digit 2. My inputs are binary images. My aim is ...
3
votes
0answers
91 views

Duda, Hart, Stork No Free Lunch Discussion

Please see this question regarding Duda, Hart, and Stork's No Free Lunch Thm Discussion Hi all, I was having trouble understanding the description of the NFL theorem in Duda, Hart, and Stork. My ...
3
votes
0answers
63 views

What is the posterior probability of the data given the model used for model averaging with Bayesian Logistic Regression?

I am trying to learn about Bayesian Model Averaging using Bayesian Logistic Regression (Genkin, A., Lewis, D. D., & Madigan, D. (2007). Large-scale Bayesian logistic regression for text ...
3
votes
0answers
37 views

Record linkage when sources have different fields

I have read a little about record linkage, but it seems to me that a requirement is that all fields in both sources can be compared. For example, with sources A and B, an assumption is that we can ...
3
votes
0answers
98 views

How to estimate False Discovery Rate from p-value distribution?

I have learned many models and I calculated p-values for the cross-validation errors. I want to select significant models based on the false discovery rate (FDR). How can I estimate the FDR from ...
3
votes
0answers
60 views

Machine learning with ordered labels

The usual method for adapting binary classifiers like various SVMs to multilabel data is one-vs-all, which assumes that labels are independent and in case of a prediction error we don't care what ...