Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms (1)

2
votes
2answers
35 views

Machine Learning Methods for Binary Classification

I was hoping to get a nice list of alternatives to logistic regression and decision trees for binary classification ("Yes vs. No" or "Cured vs. Not cured"). I am more interested in identifying the ...
3
votes
2answers
473 views

Clustering of points based on vector feature similarities in R

I have as an input a number of points that I need to partition into clusters. Each point has a number of features that are ideally to be used to find the similarity between each point and the others. ...
0
votes
0answers
9 views

Why does Alex Graves use a mixture model with his RNN instead of just directly predictive real values?

Alex Graves created a model to generate hand writing sequences which use an LSTM (kind of Recurrent Neural Network) to predict the parameters for an mixture model. The mixture model is then used to ...
0
votes
0answers
4 views

Intuition behind using boolean frequencies in tf and multiplicities in tfidf in gensim

I have a general question about the intuition behind the implementation of the tfidf model in gensim. I understand that by default gensim uses as term frequencies Boolean frequencies, i.e. 1 if term ...
0
votes
0answers
133 views

Stacked Generalization Ensemble Algorithm for regression

I am using stacked generalization(Rupert 1992) for combining multiple(8) heterogeneous base learners for regression. What I understand from the pseudo codes that Train the 8 learners on 8 instances ...
0
votes
0answers
14 views

classification error in supervised learning problem

Suppose you have a supervised learning project where it is not easy to check whether the value you predicted is correct or not. So, in this case, does it still make sense to talk about the ...
5
votes
1answer
424 views

Choosing an appropriate minibatch size for stochastic gradient descent (SGD)

Is there any literature that examines the choice of minibatch size when performing stochastic gradient descent? In my experience, it seems to be an empirical choice, usually found via ...
0
votes
1answer
76 views

Introduction to recurrent neural networks?

I have two questions: 1- What are the applications of recurrent neural networks? 2- Can you recommend some good resources/papers/tutorials that introduce recurrent neural networks?
0
votes
0answers
70 views
+50

Recurrent Neural Network (LSTM) Questions

I am coding an RNN-LSTM module, and I have the following questions: 1) In non-linear input like one-hot-encoded words, you use an extra sign (like <eol>) at ...
0
votes
0answers
14 views

A book on digital signal processing and control theory for machine learners?

I noticed that digital signal processing (DSP) and control theory (CT) are essential to master machine learning, at least I saw that many great machine learning scholars are experts in these two ...
1
vote
1answer
249 views

How to understand the label-bias problem in HMM?

How can I understand the label-bias problem in Hidden Markov Models? And why is CRF able to solve this problem?
2
votes
1answer
36 views

Variational Inference: good inference but ELBO decreases instead of increasing

I am playing with Variational Inference for clustering within a mixture of Gaussians. My first implementation seems to work fine (this is for the geyser dataset): ...
0
votes
0answers
16 views

[CNN]: Predict label of one single image using DeepLearnToolbox [on hold]

I am using DeepLearnToolbox to do CNN (Convolutional Neural Networks). I have computed my network successfully and I've seen my accuracy, but my question is: How can I query one single image into ...
0
votes
0answers
13 views

What are are some common concepts regarding optimizing a algorithm with forecasted input?

I am a programmer working on a problem where I need to make an optimization based on forecasted data. I feel kinda lost, and would like to find some statistics/machine learning articles on similiar ...
0
votes
0answers
8 views

**Kappa measure in Random Forests** [on hold]

Following is the detailed summary of trained model by Random Forests: ...
6
votes
1answer
2k views

Difference between a SVM and a perceptron

I am a bit confused with the difference between an SVM and a perceptron. Let me try to summarize my understanding here, and please feel free to correct where I am wrong and fill in what I have missed. ...
3
votes
2answers
194 views

Loss for Kernel Ridge Regression

Is $||Y-X\beta||_2^2 + \lambda\beta^T K\beta$ , the standard loss-function in kernel ridge regression, or is it different? Also, is the gaussian kernel a standard choice used for the kernel, in ...
2
votes
1answer
85 views

R: Finding relationships between 2 variables to determine any patterns in data

I am working on finding relationships/patterns between 2 variables (Type_A, Type_B). ...
0
votes
1answer
27 views

How to tune parameters through cross-validation without grid search?

There are actually lots of questions about parameter tuning through cross-validation. I have read some of them, e.g. this one. I, however, still can't understand the details of the process. Here are ...
0
votes
0answers
26 views

How to model the relation between two variables based on other common relations?

I am trying to model an inference problem, but it doesn't seem to readily fit with the algorithms we usually hear about. I am hoping that I have missed something, and someone hear can point out that ...
2
votes
0answers
34 views

How to find a cost function with only a statistical measure of success?

Using the U.S.A. as a loose analogy, we have search algorithms that find the names and number of States adjacent to a given State (containing a selected city). The goal is to minimize the number of ...
0
votes
4answers
370 views

A multi-label classification for tagging short text

I am fairly new in the area of text mining and want to practice my skills a little. I have the following task at hand which I want to work on. I have a large list of short texts (~100.000) and every ...
0
votes
0answers
9 views

How to calculate the probability of a sample to get a specific pattern, given all samples and features?

I have a data aggregation tool, which collects information on many samples (clusters of molecules in this case). So it can create a sparse matrix of NxM binary values (N answers of 0 or 1 for M ...
4
votes
2answers
265 views

Is a Gaussian-Gaussian RBM just a linear model?

The 'conventional' configuration of RBMs are Binary-Binary and Gaussian-Binary (and sometimes Binary-Gaussian) units. Although it is possible for both the visible and hidden units to be gaussian, ...
2
votes
1answer
45 views

How to simulate more data for machine learning?

I am attempting to analyze a small dataset using machine learning (SVM, binary problem). There are $103$ samples and $215$ variables (all variables are real numbers). Some of the variables (around ...
1
vote
2answers
52 views

How to use cross-validation with regularization?

I think I understand each of these concepts (cross-validation, regularization) independently, but I'm not quite clear on how they can be put together in practice. Loosely speaking, in ...
0
votes
1answer
196 views

Softmax regression or $K$ binary logistic regression

For a multi-class classification problem, we can use $K$ binary logistic classifiers, or one softmax regression classifier, so how to make the choice between the two? IMHO, the $K$ binary logistic ...
0
votes
0answers
20 views

Error function of two layer neural network

Let's say we have a two layer neural network with $\theta^1$ and $\theta^2$ as the layer units. Now ${\delta J \over \delta \theta^2}$ is convex and has a global minima for each unit. But what is the ...
1
vote
2answers
43 views

Is validation set always necessary?

Lets say I did the following steps: Used some separate development set to select some features. Decided a priori to use only one learning algorithm (SVM) with only default parameter values. Trained ...
1
vote
0answers
15 views

Different Methods for clustering skills in text

Consider a talent pool in which each member has some set of skills. Some of these talent are submitted to orders as potential candidates of which one is selected. It is reasonable to assume that the ...
3
votes
1answer
401 views

How to combine the responses of two sensors?

I have two sets of responses from two different sensors. In each set, the first column is distance measured in feet, and the second column is the response of the sensor. Sensor A has response values ...
0
votes
1answer
31 views

What are the differences between autoencoder and t-SNE?

As far as I know, both autoencoder and t-SNE are used for nonlinear dimension reduction. What are the differences between them and why should I use one versus another? thanks!
3
votes
2answers
45 views

Recurrent vs Recursive Neural Networks: Which applies better for NLP?

So, we have Recurrent Neural Networks and Recursive Neural Networks. Both are usually denoted by the same acronym: RNN. According to Wikipedia, Recurrent NN are in fact Recursive NN, but I don't ...
0
votes
1answer
33 views

Opportunities in machine learning and computational intelligence [on hold]

I'm not sure this is the right site to post my question. If not, please direct me to the right one. I'm interested in machine learning and computational intelligence. I've spent the last year of my ...
2
votes
0answers
33 views

Estimating probability distribution function of data stream

Although a similar question exists, I couldn't find my answer. I'm not a statistician hence please neglect if some terminologies aren't correct and let me know if I am interpreting something wrong. ...
0
votes
0answers
22 views

Data augmentation techniques for general datasets?

In many machine learning applications, the so called data augmentation methods have allowed building better models. For example, assume a training set of $100$ images of cats and dogs. By rotating, ...
0
votes
1answer
50 views

How to stack a convolutional autoencoder?

I am trying to figure out as to how to stack a convolutional autoencoder (CAE)? Consider a convolutional autoencoder (CAE) (using MNIST data, 28x28 input dimensions): ...
40
votes
13answers
6k views

Machine learning cookbook / reference card / cheatsheet?

I find resources like the Probability and Statistics Cookbook and The R Reference Card for Data Mining incredibly useful. They obviously serve well as references but also help me to organize my ...
3
votes
1answer
73 views

How do you transform a decision boundary in the angle kernel to the original space?

Say I have training data $S_n$ and each point is of the form $x = \langle x_1 , x_2 \rangle$ in the original space (i.e. $x^{(i)} \in \mathbb{R}^2$). I was considering the following kernel: $$ ...
0
votes
0answers
23 views

Does this simple anomaly detection algorithm work for > 100 features?

In Andrew Ng's Coursera class on Machine Learning, we learned to use a Gaussian distribution $p(x) = \prod_{j=1}^n p(x_j,\mu_j,\sigma_j^2)$ to detect anomalous examples when $p(x) < \epsilon$ ...
0
votes
1answer
32 views

Neural Networks and Numeric Prediction

I'm new to machine learning and am trying to write a simple neural network that uses back-propagation. Now, so far I've successfully implemented my neural network to learn a boolean function. So for ...
2
votes
1answer
51 views
+150

Standardization for regularized, sparse hashed logistic regression

As the question states, I'm fitting large, sparse logistic regressions (with hashed interactions, a la vowpal wabbit) for a machine learning system. The features are on different scales, and I'm a ...
0
votes
2answers
33 views

No regularisation term for bias unit in neural network

According to this tutorial on deep learning, weight decay (regularization) is not usually applied to the bias terms b why? What is significance (intuition) behind it?
0
votes
0answers
31 views

Feature extraction based on correlations

I have a small problem regarding feature extraction with correlation. I have divided my question in four parts hoping that somebody can help me. I have a dataset consisting of fMRI images. Each image ...
0
votes
1answer
20 views

Intuition behind sparsity in over-complete sparse auto-encoders

I am trying to get a grasp of the intuition behind the sparse representation used in over-complete auto-encoders. One piece of text that offers a somewhat intuitive explanation is from ...
4
votes
2answers
72 views

Sequential pattern mining on single sequence

Can someone give me a hint about a good approach to find a frequent patterns in a single sequence. For example there is the single sequence ...
1
vote
1answer
132 views

How to validate sentiment classification and compare different algorithms

I need to compare SVM and NB about sentiment classification by evaluating accuracy, precision and recall measures. I have 1500 manually classified documents, and I would know which is the best way to ...
1
vote
1answer
273 views

Observation symbols for training a set of HMMs

If we are to classify 2 separate classes/actions using HMMs, we design 2 separate HMMs (one for each class). Do they share same set or a different set of observations-symbols for each of the HMM? If ...
11
votes
3answers
283 views

How to intuitively explain what a kernel is?

Many machine learning classifiers (e.g. support vector machines) allow one to specify a kernel. What would be an intuitive way of explaining what a kernel is? One aspect I have been thinking of is ...
1
vote
3answers
161 views

Linearly dependent features

I have a matrix A of 1000 observations (rows) and 100 features (cols). I would like to find: Linearly dependent features so that I can remove them and simplify the problem. rank(A) gives me 88, ...