Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms (1)

10
votes
0answers
416 views

Link Anomaly Detection in Temporal Network

I came across this paper that uses link anomaly detection to predict trending topics, and I found it incredibly intriguing: The paper is "Discovering Emerging Topics in Social Streams via Link Anomaly ...
7
votes
0answers
56 views

Are contours $h^{-1}(y)$ interesting features of a function $h:X\to \mathbb R^n$ obtained by regression?

I assume a general setup of regression, that is, a continuous function $h_\theta:X\to \mathbb R^n$ is chosen from a family $\{h_\theta\}_\theta$ to fit given data $(x_i,y_i)\in X\times \mathbb R^n, ...
7
votes
0answers
93 views

Speed, computational expenses of PCA, LASSO, elastic net

I am trying to compare computational complexity / estimation speed of three groups of methods for linear regression as distinguished in Hastie et al. "Elements of Statistical Learning" (2nd ed.), ...
7
votes
0answers
127 views

State of art streaming learning

I have been working with large data sets lately and found a lot of papers of streaming methods. To name a few: Follow-the-Regularized-Leader and Mirror Descent: Equivalence Theorems and L1 ...
7
votes
0answers
169 views

Are all models useless? Is any exact model possible — or useful?

This question has been festering in my mind for over a month. The February 2015 issue of Amstat News contains an article by Berkeley Professor Mark van der Laan that scolds people for using inexact ...
7
votes
0answers
248 views

Convolutional neural networks: Aren't the central neurons over-represented in the output?

[This question was also posed at stack overflow] The question in short I'm studying convolutional neural networks, and I believe that these networks do not treat every input neuron ...
6
votes
0answers
78 views

Word embedding algorithms in terms of performance

I'm trying to embed roughly 60 million phrases into a vector space, then calculate the cosine similarity between them. I've been using sklearn's CountVectorizer ...
6
votes
0answers
1k views

Confusion with Vowpal Wabbit's multiple-pass behavior when performing ridge-regression

I have encountered many peculiarities/misunderstandings of Vowpal Wabbit when trying to do online multiple-pass learning. Specifically, I need to solve a Ridge Linear regression problem, with ...
6
votes
0answers
1k views

Anomaly Detection with Dummy Features (and other Discrete/Categorical Features)

tl;dr What is the recommended way to deal with discrete data when performing anomaly detection? What is the recommended way to deal with ...
6
votes
0answers
645 views

Post processing random forests using regularised regression: what about bias?

I have been playing around with post processing the results of the random forest for regression machine learning algorithm in order to try and do better than the default mean of all trees prediction. ...
6
votes
0answers
1k views

Maximum entropy classifier and sentiment analysis

I am doing a project work in sentiment analysis (on Twitter data) using machine learning approach. In order to find the 'best' way to this I have experimented with naive Bayesian and maximum entropy ...
6
votes
0answers
188 views

Self-organizing maps: fuzzy input?

as my first post I would like to know if there are SOM implementations (preferably R) available that accept fuzzy input. That is, I have data in which some nominal features are spread out between a ...
5
votes
0answers
93 views

Understanding linear projection in “The Elements of Statistical Learning”

In the book "The Elements of Statistical Learning" in chapter 2 ("Linear models and least squares; page no: 12"), it is written that In the (p+1)-dimensional input-output space, (X,Y) represent a ...
5
votes
0answers
48 views

Why is the variable importance metric suggested by Breiman specific only to random forests?

In the Random Forest paper they describe a nice way of measuring a variable importance - take your validation data, measure error rate, permute the variable and re-measure error rate. Question - why ...
5
votes
0answers
304 views

State of the Art versions of Generalized Additive Models

Generalized Additive Models [Trevor Hastie and Robert Tibshirani 86] was well received with over 1335 Citations. I am also aware of the popular(?) version of GAM - the Multivariate Adaptive ...
5
votes
0answers
184 views

Explanation for large difference in SVM and Naive bayes results

I have a dataset with 389 data evenly distributed into 6 classes. Each data has 1024 features. So my dimension is much larger than my observation data. I have tried to see some common classifiers on ...
5
votes
0answers
176 views

Rademacher complexity of logistic regression

Consider logistic regression. We have the logistic loss function, $\phi: R\rightarrow [0,1], \phi(u)=\log(1+\exp(-u))$, which is Lipschitz, and we have the linear function class $F=\{f_w:R^d ...
5
votes
0answers
127 views

Expectation Maximization Clarification

I found very helpful tutorial regarding EM algorithm. The example and the picture from the tutorial is simply brilliant. Related question about calculating probabilities how does expectation ...
5
votes
0answers
115 views

Compressed sensing: Optimization in $L_1$ norm and total variation with fourier coefficients

I'm reading the article Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information (Candes, Romberg and Tao, 2004). In this article they are talking ...
5
votes
0answers
583 views

Manifold regularization using laplacian graph in SVM

I'm trying implement Manifold Regularization in Support Vector Machines (SVMs) in Matlab. I'm following the instructions in the paper by Belkin et al.(2006), there's the equation in it: $f^{*} = ...
5
votes
0answers
103 views

Graphical nominal model

Suppose I have a set of $k$ matrices. $$ \mathbb D = A_1,A_2,...,A_k $$ Each column of $A$ is categorical vector. $$ A = v_1,v_2,...,v_n $$ I want to find the mapping $$ f: A ...
4
votes
0answers
63 views

How to estimate a probability distribution

Suppose I want to estimate a probability distribution, is it common practice to simply fit a function to a frequency histogram? So in my work, I am training a classifier, the performance of which is ...
4
votes
0answers
163 views

Ideal statistical or machine learning technique to model highly cross-correlated data

I'm trying to build a model that can predict streamflow for an alpine (snowmelt-fed) watershed using snow albedo (roughly, the energy reflectance of the snow) data. Albedo controls the melt of the ...
4
votes
0answers
126 views

Using standard machine learning tools on left-censored data

I'm developing a forecasting application whose purpose is to allow an importer to forecast demand for its products from its customer network of distributors. Sales figures are a pretty good proxy for ...
4
votes
0answers
148 views

Model selection in offline vs. online learning

I've been trying to learn more about online learning lately (it's absolutely fascinating!), and one theme that I haven't been able to get a good grasp on is how to think about model selection in ...
4
votes
0answers
72 views

Vapniks proof of the basic lemma

In his book Statistical Learning Theory (1998), Vladimir Vapnik proves an inequality needed to prove a bound on the risk for indicator loss functions. Theorem 4.1 on page 133 he derives the following ...
4
votes
0answers
587 views

How to perform hypothesis testing for comparing different classifiers

I am trying to classify a small dataset (around 500 records) into two classes. I used various methods like SVM, Naive Bayes and k-nn classifier. Now, I would like to set the results from one of the ...
4
votes
0answers
129 views

What kind of plot am I looking at?

I stumbled on to these following two slides (slides 21 & 22 on a machine learning tutorial found here): The first is obviously an $x,y$ scatterplot of height and weight. But what is the ...
4
votes
0answers
72 views

How to model the distribution of a word game in order to find correlation between demographics and chosen words

I have an experiment (in the form of a word game) whereby people are asked to choose a set of words to describe associations with a topic with the aim of having another person guess the topic. I ...
4
votes
0answers
275 views

detecting circadian rhythm in a time series

I have a sensor that can detect minute changes in distance. It produces a time series. I would like to point it at people and detect things like their sleeping pattern. How would one build a system ...
4
votes
0answers
206 views

What are the most popular domain adaptation methods (for transfer learning)?

I understand supervised and unsupervised learning well, and would be able to identify some 'basic' examples of, for example, supervised classifcation as: SVMs Random Forests Logistic Regression ...
4
votes
0answers
405 views

Incremental learning methods in R

I am looking for some libraries in R that can do incremental learning (also called online or sequential learning). The use case of such learning in comparison to traditional batch methods would be to ...
4
votes
0answers
84 views

How to learn similarity of typed/attributed graphs?

I have a question for graph machine learning gurus :). For this project I'm working on, I need to be able to learn similarity between typed graphs. By typed I mean that every vertex and every edge of ...
4
votes
0answers
518 views

Calculating VC-dimension of a neural network

If I have some fixed non-recurrent (DAG) topology (fixed set of nodes and edges, but the learning algorithm can vary the weight on the edges) of sigmoid neurons with $n$ input neurons which can only ...
4
votes
0answers
216 views

Category selection for text classification

It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
3
votes
0answers
18 views

Propagating uncertainties using random forest out-of-bag accuracy estimates

Let's say I train a random forest on some data and get an out-of-bag accuracy estimate of 90%. I then predict a quantity using this trained forest. What should be the uncertainty I give to that ...
3
votes
0answers
29 views

Is dimensional reduction using Autoencoders possible with a small sample size?

I have a data set that is not too big but high dimensional, let say 10000 dimensional. I want to use an autoencoder to extract relevant features (clusters) in the data. Usually when I have seen ...
3
votes
0answers
31 views

Cut off point with three categories

I have a categorical variable called severity with three categories (low, medium, high) and another numerical variable call ZX that can take values from 0 to 10. I want to find the cut off points of ...
3
votes
0answers
432 views

Dealing with auxiliary random variables for Mean-Field Variational Inference in Bayesian Poisson factorization

I am studying as a part of a class assignment a recent paper on Poisson factorization. Some points of the paper regarding the usage of some auxiliary variables are not clear to me. I would like to ...
3
votes
0answers
42 views

Bayesian interpretation of the repeated sampling principle

My question is philosophical rather than practical, and I will try to explain it through an example: Consider a Kaggle competition. All these contests have a similar structure: A "train dataset" is ...
3
votes
0answers
31 views

Selection of Loss Function for Weather Insurance

I am using machine learning to predict agricultural yields using weather variables as inputs. I have been thinking about what loss function to optimize. I have been using RMSE thus far. The ...
3
votes
0answers
34 views

What is the acceptable event rate to use ROC-AUC instead of precision-recall curve?

It says here However, when dealing with highly skewed datasets, Precision-Recall (PR) curves give a more informative picture of an algorithm's performance. My question is; What is the common ...
3
votes
0answers
77 views

Which Machine Learning book to choose (APM, MLAP or ISL)?

I'm searching a book as a referesher in machine learning (I have taken a lecture in machine learning some times ago). I will be applying machine learning in a project. I have searched a lot of books ...
3
votes
0answers
36 views

Is anyone using the last but one layer of a deep neural networks for classification or search?

According to the paper "Return of the Devil in the Details: Delving Deep into Convolutional Networks" the last but one layer in a deep neural network trained on a very large dataset can be used as ...
3
votes
0answers
46 views

What key information should be included in an academic paper that uses machine learning?

Imagine you are reviewing a manuscript that describes application of a supervised machine learning algorithm (e.g. SVM, CART, logistic regression, random forest etc.) to predict a binary output. ...
3
votes
0answers
49 views

Are interactions only useful in the context of regression?

I have always read the term interaction in the context of regression. Should we also consider interactions with different models e.g. knn or svm? If there are $50$, $100$ or even more features and ...
3
votes
0answers
48 views

How can I estimate the influence/significance of the every observation on classification?

There are many ways to estimate the significance of the features on the classification model. But how I can estimate the influence of the every observation on the classification quality? My thinking ...
3
votes
0answers
45 views

What to do when LDA over-fits

I'm looking for guidance on ways to improve my test set prediction accuracy when using Linear Discriminant Analysis (LDA). I have a matrix of ~10K rows x 24K columns. Of the 24K features, 4 represent ...
3
votes
0answers
53 views

How to properly develop a machine learning model for a poker game?

I've created an annotation for poker games, similar to chess games. After compiling information from thousands of games, I want to use this large data set for machine learning. To simplify, let's ...
3
votes
0answers
81 views

How to build “supervised clustering” for neural networks?

I'm confused as to what the output would be. Consider the "blind source separation" problem. Let's say I have a ton of training examples where the input is the final cacophony of sounds as a sound ...