Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms (1)

11
votes
0answers
453 views

Link Anomaly Detection in Temporal Network

I came across this paper that uses link anomaly detection to predict trending topics, and I found it incredibly intriguing: The paper is "Discovering Emerging Topics in Social Streams via Link Anomaly ...
9
votes
0answers
176 views

Speed, computational expenses of PCA, LASSO, elastic net

I am trying to compare computational complexity / estimation speed of three groups of methods for linear regression as distinguished in Hastie et al. "Elements of Statistical Learning" (2nd ed.), ...
8
votes
0answers
64 views

Are contours $h^{-1}(y)$ interesting features of a function $h:X\to \mathbb R^n$ obtained by regression?

I assume a general setup of regression, that is, a continuous function $h_\theta:X\to \mathbb R^n$ is chosen from a family $\{h_\theta\}_\theta$ to fit given data $(x_i,y_i)\in X\times \mathbb R^n, i=...
8
votes
0answers
158 views

State of art streaming learning

I have been working with large data sets lately and found a lot of papers of streaming methods. To name a few: Follow-the-Regularized-Leader and Mirror Descent: Equivalence Theorems and L1 ...
8
votes
0answers
335 views

Convolutional neural networks: Aren't the central neurons over-represented in the output?

[This question was also posed at stack overflow] The question in short I'm studying convolutional neural networks, and I believe that these networks do not treat every input neuron (pixel/parameter)...
8
votes
0answers
2k views

Confusion with Vowpal Wabbit's multiple-pass behavior when performing ridge-regression

I have encountered many peculiarities/misunderstandings of Vowpal Wabbit when trying to do online multiple-pass learning. Specifically, I need to solve a Ridge Linear regression problem, with ...
6
votes
0answers
46 views

Restricted Boltzmann Machine : how is it used in machine learning?

Background: Yes, Restricted Boltzmann Machine (RBM) CAN be used to initiate the weights of a neural network. Also it CAN be used in a "layer-by-layer" way to build a deep belief network (that is, to ...
6
votes
0answers
1k views

Anomaly Detection with Dummy Features (and other Discrete/Categorical Features)

tl;dr What is the recommended way to deal with discrete data when performing anomaly detection? What is the recommended way to deal with ...
6
votes
0answers
701 views

Post processing random forests using regularised regression: what about bias?

I have been playing around with post processing the results of the random forest for regression machine learning algorithm in order to try and do better than the default mean of all trees prediction. ...
6
votes
0answers
1k views

Maximum entropy classifier and sentiment analysis

I am doing a project work in sentiment analysis (on Twitter data) using machine learning approach. In order to find the 'best' way to this I have experimented with naive Bayesian and maximum entropy ...
6
votes
0answers
196 views

Self-organizing maps: fuzzy input?

as my first post I would like to know if there are SOM implementations (preferably R) available that accept fuzzy input. That is, I have data in which some nominal features are spread out between a ...
5
votes
0answers
70 views

Why is the variable importance metric suggested by Breiman specific only to random forests?

In the Random Forest paper they describe a nice way of measuring a variable importance - take your validation data, measure error rate, permute the variable and re-measure error rate. Question - why ...
5
votes
0answers
84 views

Denoising Autoencoders weights at test time

When using masking noise whilst training Denoising Autoencoders should weights be increased at test time proportional to the masking rate, as in Dropout?
5
votes
0answers
408 views

State of the Art versions of Generalized Additive Models

Generalized Additive Models [Trevor Hastie and Robert Tibshirani 86] was well received with over 1335 Citations. I am also aware of the popular(?) version of GAM - the Multivariate Adaptive ...
5
votes
0answers
246 views

Explanation for large difference in SVM and Naive bayes results

I have a dataset with 389 data evenly distributed into 6 classes. Each data has 1024 features. So my dimension is much larger than my observation data. I have tried to see some common classifiers on ...
5
votes
0answers
220 views

Rademacher complexity of logistic regression

Consider logistic regression. We have the logistic loss function, $\phi: R\rightarrow [0,1], \phi(u)=\log(1+\exp(-u))$, which is Lipschitz, and we have the linear function class $F=\{f_w:R^d \...
5
votes
0answers
183 views

Model selection in offline vs. online learning

I've been trying to learn more about online learning lately (it's absolutely fascinating!), and one theme that I haven't been able to get a good grasp on is how to think about model selection in ...
5
votes
0answers
150 views

Expectation Maximization Clarification

I found very helpful tutorial regarding EM algorithm. The example and the picture from the tutorial is simply brilliant. Related question about calculating probabilities how does expectation ...
5
votes
0answers
116 views

Compressed sensing: Optimization in $L_1$ norm and total variation with fourier coefficients

I'm reading the article Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information (Candes, Romberg and Tao, 2004). In this article they are talking ...
5
votes
0answers
634 views

Manifold regularization using laplacian graph in SVM

I'm trying implement Manifold Regularization in Support Vector Machines (SVMs) in Matlab. I'm following the instructions in the paper by Belkin et al.(2006), there's the equation in it: $f^{*} = ...
5
votes
0answers
103 views

Graphical nominal model

Suppose I have a set of $k$ matrices. $$ \mathbb D = A_1,A_2,...,A_k $$ Each column of $A$ is categorical vector. $$ A = v_1,v_2,...,v_n $$ I want to find the mapping $$ f: A \...
4
votes
0answers
33 views

What are some adaptive machine learning techniques that cater for data that may change slightly but is still correct?

Are there suitable machine learning techniques that may be applied to a continual stream of data and update its models for data that it believes to be different to the most representative case but ...
4
votes
0answers
90 views

Ensemble Learning: Why is Model Stacking Effective?

Recently, I've become interested in model stacking as a form of ensemble learning. In particular, I've experimented a bit with some toy datasets for regression problems. I've basically implemented ...
4
votes
0answers
69 views

How to estimate a probability distribution

Suppose I want to estimate a probability distribution, is it common practice to simply fit a function to a frequency histogram? So in my work, I am training a classifier, the performance of which is ...
4
votes
0answers
192 views

Ideal statistical or machine learning technique to model highly cross-correlated data

I'm trying to build a model that can predict streamflow for an alpine (snowmelt-fed) watershed using snow albedo (roughly, the energy reflectance of the snow) data. Albedo controls the melt of the ...
4
votes
0answers
175 views

Using standard machine learning tools on left-censored data

I'm developing a forecasting application whose purpose is to allow an importer to forecast demand for its products from its customer network of distributors. Sales figures are a pretty good proxy for ...
4
votes
0answers
221 views

What enforces features diversity in RBM?

I'm working on an implementation of a Restricted Boltzman Machine (RBM). I made some tests on the MNIST dataset trying to learn a representation of the digit 2. My inputs are binary images. My aim is ...
4
votes
0answers
74 views

Vapniks proof of the basic lemma

In his book Statistical Learning Theory (1998), Vladimir Vapnik proves an inequality needed to prove a bound on the risk for indicator loss functions. Theorem 4.1 on page 133 he derives the following ...
4
votes
0answers
134 views

What kind of plot am I looking at?

I stumbled on to these following two slides (slides 21 & 22 on a machine learning tutorial found here): The first is obviously an $x,y$ scatterplot of height and weight. But what is the ...
4
votes
0answers
75 views

How to model the distribution of a word game in order to find correlation between demographics and chosen words

I have an experiment (in the form of a word game) whereby people are asked to choose a set of words to describe associations with a topic with the aim of having another person guess the topic. I ...
4
votes
0answers
248 views

Universal Approximation Theorem — Neural Networks

I have posted this question elsewhere--MSE-Meta, MSE, TCS, MetaOptimize. Previously, no one had given a solution. But now, here is a really excellent and comprehensive answer. Universal approximation ...
4
votes
0answers
304 views

detecting circadian rhythm in a time series

I have a sensor that can detect minute changes in distance. It produces a time series. I would like to point it at people and detect things like their sleeping pattern. How would one build a system ...
4
votes
0answers
228 views

What are the most popular domain adaptation methods (for transfer learning)?

I understand supervised and unsupervised learning well, and would be able to identify some 'basic' examples of, for example, supervised classifcation as: SVMs Random Forests Logistic Regression ...
4
votes
0answers
155 views

Using taxonomic levels as factors in random forests: does it make sense? Is it needed?

I want to test the effect of a set of predictors (ecological and morphological factors) on a categorical response variable (an animal behaviour). As far as I've read, random forests do not make ...
4
votes
0answers
472 views

Incremental learning methods in R

I am looking for some libraries in R that can do incremental learning (also called online or sequential learning). The use case of such learning in comparison to traditional batch methods would be to ...
4
votes
0answers
92 views

How to learn similarity of typed/attributed graphs?

I have a question for graph machine learning gurus :). For this project I'm working on, I need to be able to learn similarity between typed graphs. By typed I mean that every vertex and every edge of ...
4
votes
0answers
603 views

Calculating VC-dimension of a neural network

If I have some fixed non-recurrent (DAG) topology (fixed set of nodes and edges, but the learning algorithm can vary the weight on the edges) of sigmoid neurons with $n$ input neurons which can only ...
4
votes
0answers
247 views

Category selection for text classification

It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
3
votes
0answers
26 views

Q-learning with Neural Networks: one output unit per action

I am using Neural Network Q-value approximation in my reinforcement learning task. The approach is exactly the same as one described in this question, however the question itself is different. In ...
3
votes
0answers
44 views
+50

How does one interpret histograms given by TensorFlow in TensorBoard?

I recently was running and learning tensor flow and got a few histograms that I did not know how to interpret. Usually I think of the height of the bars as the frequency (or relative frequency/counts)....
3
votes
0answers
44 views

Perceptron trained on time series always predicting the same answer

Using the model from theano's tutorial, I'm training a 3-layers perceptron with log returns over a very large dataset (~55,000 points). The output's layer contains two neurons, one for each of the ...
3
votes
0answers
35 views

K-fold cross-validation for time series with dynamic target variable (Scikit)

I would like to do a K-fold cross-validation on time series data (market data) with a two class classification target. My test folds must be forward looking and of a fixed size ...
3
votes
0answers
32 views

Is it ok to use symmetric loss function when evaluation metric is asymmetric?

I completely understand that it's ok to use a loss function different from the evaluation metric. For example, accuracy isn't computationally feasible to optimize directly since it's not ...
3
votes
0answers
35 views

classifier performing well in leave-one-out cross-validation but not k-fold

I am building a classifier and have over 1 million features to choose from. I implemented penalized regression, aka, lasso regression, followed by recursive feature selection in order to select ...
3
votes
0answers
77 views

What is the intuition behind a Long Short Term Memory (LSTM) recurrent neural network?

The idea behind Recurrent Neural Network (RNN) is clear to me. I understand it in the following way. We have a sequence of observations ($\vec o_1, \vec o_2, \dots, \vec o_n$) (or, in other words, ...
3
votes
0answers
16 views

Extract a sub-distribution with specific characteristics

Imagine that I have a distribution of some data. For example a distribution of natural numbers [1, 10, 45, 89, 12, 9, 4, 100]. I characterize such distribution with 2 features: the mean = 33.75 and ...
3
votes
0answers
22 views

Does this pattern indicate over-fitting in machine learning?

I am working on a diagnostics project, and trying to improve the performance of a classifier(s). We have over a million features to choose from, so feature selection is a real challenge. To look ...
3
votes
0answers
23 views

VC dimension of a quadratic binary classifier

Assume we have a set of training data $\{\boldsymbol{x}_i, y_i\}_{i=1}^n$ from $\mathbb{R}^2 \times \{-1, 1\}$. The hypotheses $\mathcal{H} $ are all classifiers with the form $\hat{y} = \text{sign}(\...
3
votes
0answers
57 views

Understanding calibrating probabilities using R

I am trying to understand R's calibration(package:caret) function. My main interest is binary classification. Calibration function is used for plotting true ...
3
votes
0answers
26 views

Propagating uncertainties using random forest out-of-bag accuracy estimates

Let's say I train a random forest on some data and get an out-of-bag accuracy estimate of 90%. I then predict a quantity using this trained forest. What should be the uncertainty I give to that ...