Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms

1
vote
1answer
483 views

Decision boundaries and Gaussian density functions

This is for my hw, and if anyone can solve the first part of the question it will be great. Here is the question: Assume a two-class problem with equal a priori class probabilities and Gaussian ...
1
vote
1answer
34 views

Can a machine learning algorithm be evaluated based on a random sample?

I am trying to evaluate how well (or bad) a semi-supervised algorithm is performing on a given dataset. The algorithms assigns one of 10 labels to each data point. The dataset is huge, and it's not ...
8
votes
2answers
626 views

Are there any libraries available for CART-like methods using sparse predictors & responses?

I'm working with some large data sets using the gbm package in R. Both my predictor matrix and my response vector are pretty sparse (i.e. most entries are zero). I was hoping to build decision trees ...
2
votes
3answers
52 views

Does using a kernel function make the data linearly separable?

I'm reading about SVM and I learned that we use a kernel function so the data become linearly separable in the high dimensional (vector?) space. But then I also learned that they use the soft-margin ...
1
vote
1answer
27 views

Dealing with different time series data in Machine Learning

I am trying to create a stock market model based on fundamental variables for the US economy. I am using R. Some of the variables I am looking to include are: GDP, Unemployment Rate, Initial Claims, ...
2
votes
1answer
83 views

What are the main differences between classical and Gibbs sampling Latent Dirichlet Allocations?

In these weeks I have been studying the classical Latent Dirichlet Allocation (LDA) algorithm by David Blei and colleagues (2003), and the LDA variant based on Gibbs sampling introduced by Tom ...
2
votes
1answer
115 views

Gaussian Process Kernel and Ridge Regression

Can a Dual Ridge Regression produce the same prediction results as a Gaussian Process with a polynomial kernel $K(x,x')=(x^Tx'+1)^2$ in less time complexity (GP is $O(n^3)$ ) using Cholesky ...
0
votes
1answer
31 views

Classification Accuracy

I am classifying text based on news headlines and I am achieving accuracy up to approx 80%. I want to improve it more. But issue is that when I calculate the same with synonyms using the code below: ...
0
votes
0answers
15 views

Tutorial on Radial Basis Function Networks?

I want to learn about Radial Basis Function Neural Networks, can you please suggest a good introduction or tutorial? All the introductions I found are rather short or incomplete or so.
3
votes
1answer
143 views

Measuring and analyzing sample complexity

I recently stumbled upon the concept of sample complexity, and was wondering if there are any texts, papers or tutorials that provide: An introduction to the concept (rigorous or informal) An ...
1
vote
1answer
339 views

Training a convolutional neural network

Based on my research on convolution neural networks, every other layer in such a network has a subsampling operation, in which the resolution of the image is reduced so as to improve generalization of ...
0
votes
0answers
13 views

Feature Selection for look alike modeling using k-NN

I have a list of items(around 7000) and various parameters for each items. For each item on my list i need to identify 10 items which are similar to the item from my whole population (17 million). I ...
0
votes
1answer
23 views

Decision boundary equation of the perceptron

As I know the standard linear equation has the following form in $R^2$: $w_1 x_1 + w_2 x_2 = b$ where $b$ is the intercept with $x_2$ Also I know that a decision boundary in $R^2$ for a perceptron ...
2
votes
1answer
31 views

algorithms to pick best collection of classifiers

I have a bunch of methods that classify a binary outcome. I'm trying to figure out if some combination of those classifiers is better than any others. I'm hoping to run a bunch of methods. I've ...
0
votes
0answers
7 views

How to validate classifier (built by using MLN method)?

I have developed a method (let's call it Method X) that has a classifier function. The classifier function was built by using MLN (Markov logic network). I need to ...
6
votes
2answers
557 views

Meaning of a neural network as a black-box?

I often hear people talking about neural networks as something as a black-box that you don't understand what it does or what they mean. I actually I can't understand what they mean by that! If you ...
0
votes
1answer
84 views

What is SVM regression? Is it for regression or classification?

I'm trying to understand what is SVM regression. It's used for classification or regression? Can someone give an intuitive understanding of it?
-2
votes
0answers
37 views

Technologies behind Siri and Google Now?

I'm interested to know how Siri and Google Now works. What are the technologies behind them? I don't mean voice recognition (which is obvious) but the other stuff. Like interpreting the input and ...
1
vote
2answers
45 views

Class labels in data partitions

Suppose that one partitions the data to training/validation/test sets for further application of some classification algorithm, and it happens that training set doesn't contain all class labels that ...
0
votes
1answer
36 views

How to make stochastic gradient descent algorithm converge to the optimum?

(Background info taken from my blog) In logistic regression, the hypothesis function, which models the relationshiop between the dependent variable $P(y = 1)$ and the independent variable $X$, is : ...
0
votes
0answers
22 views

What is multi run lasso regression?

I have problem in understanding of multi-run lasso regression. Basically, I know what is lasso regression, but don't know what is multi-run lasso regression, which sometimes I see literatures. Does ...
0
votes
1answer
31 views

Performing hierarchical clustering on a large data set

I have been applying complete linkage on about 5,000 points using matlab with no problem. I want to extend this method to much more elements. It would take me a long time to process my data to test ...
2
votes
1answer
31 views

How can I evaluate the performance of a system that generates word clusters?

The word2vec tool uses deep learning to compute vector representations of words. They've mentioned that - "The word vectors can be also used for deriving word classes from huge data sets. This is ...
3
votes
1answer
126 views

Reinforcement learning of a policy for multiple actors in large state spaces

I have a real-time domain where I need to assign an action to N actors involving moving one of O objects to one of L locations. At each time step, I'm given a reward R, indicating the overall success ...
1
vote
1answer
95 views

Poorness of Kernel methods on visual pattern recegnition?

I am currently reading the recent papers mainly written by Y. Bengio [1],[2],[3]. There are very strong claims about poorness of Kernel methods on recognizing handwritings in many general cases but ...
1
vote
0answers
20 views

Are these descriptions of batch gradient descent algorithm conflicting each other?

The first one is from Andrew Ng The second one is from Francis Bach I might be a little confused, but why is there a summation of partial derivatives in the second description and none in the ...
2
votes
1answer
39 views

Gradient decay in neural networks

I read that in traditional feed-forward neural nets the gradients in the early layers decay very quickly and that this is 'a bad thing'. But I don't understand why. Can someone please explain what ...
3
votes
1answer
45 views

Why can we use entropy to measure the quality of a language model?

I am reading the < Foundations of Statistical Natural Language Processing >. It has the following statement about the relationship between information entropy and language model: ...The ...
30
votes
4answers
4k views

What does the hidden layer in a neural network compute?

I'm sure many people will respond with links to 'let me google that for you', so I want to say that I've tried to figure this out so please forgive my lack of understanding here, but I cannot figure ...
0
votes
2answers
24 views

Latent Dirichlet Allocation as input for WEKA

I am using the Weka API for my research about document classification. I wish to apply Latent Dirichelet Allocation on my dataset followed by using a classifier in Weka. However, it is not so clear to ...
3
votes
2answers
102 views

How to calculate p values in logistic regression with gradient descent algorithm

In logistic regression, the gradient descent algorithm for calculating coefficients can be described this way: Until convergence, do $$ \beta := \beta + \alpha \frac{\partial L}{\partial \beta} ...
0
votes
1answer
101 views

“Philosophical” data contamination question

I'm going through Yaser Abu-Mostafa's Learning from Data course, and I'm having some trouble getting my head around data contaimination. So we know stuff like VC-analysis can give us some gaurentee ...
1
vote
0answers
31 views

Rough estimates for training time of deep belief networks

I'm still learning about deep learning. However I'm currently interested to know if deep learning architectures scale well or not. Suppose I have a dataset with 1 million training examples, can you ...
4
votes
1answer
268 views

Variational inference for nested Chinese restaurant process

I recently read paper by Chong Wang and David M. Blei "Variational Inference for the Nested Chinese Restaurant Process". And I couldn't understand the next part (from p.5): The variational update ...
0
votes
0answers
10 views

Why feature maps are indexed by two indices?

I'm reading about convolutional neural networks. As I understood a feature map is a set of neurons (i.e like a single hidden layer in traditional ANN). So why feature maps are indexed by (i,j)? ...
0
votes
1answer
84 views

Machine learning with trinomial features

I have 100,000 students who have each answered some multiple choice questions. Given their performance I want to work out what the chances are of a particular student answering the next question ...
0
votes
1answer
87 views

Quantitative results of clustering analysis

Currently, I am doing a clustering analysis for two sets of data. One smaller dataset (about 100 data) got ground truth labels, and one larger dataset (about 2000 data) has no ground truth labels. ...
0
votes
1answer
40 views

How to set the radius value in self organizing map?

I'm training the self organizig map, I need to set the value for the radius of it. is there any method to find the optimum radius size ?
0
votes
1answer
19 views

Updating set of probabilities for sampling with features importance

I'm currently working on some algorithm and I'm kinda out of idea for a problem I'm trying to tacle. Basically I'm trying to subsample the features of a dataset. I want to subsample that given this ...
0
votes
1answer
25 views

Standardising the weights generated from feedforward back propagation NN

I have binary data input to NN. The weights generated by NN are not normalized, i.e., the weights are not the same for every run of the algorithm. How to standardize these weights?
1
vote
1answer
52 views

How is AUC of decision tree calculated?

I have a dataset which only has one continuous variable, and I try to use decision tree algorithm to build a model which classify the +ve and -ve label from the dataset. I run 10-fold ...
2
votes
1answer
225 views

Weighted covariance matrix using kernels

I would like to create a weighted covariance matrix (say 5 variables) using 3 different time points where the weights come from a kernel function (can be normal, triangular, etc.) but I'm not ...
1
vote
1answer
34 views

Why features compression is good?

I'm reading about deep learning and that in principles it's a features compression technique and that is why it works. Now my question is why compressing features from 200 or so into 4 is better? How ...
1
vote
2answers
97 views

Not very known but effective data mining algorithms?

I recently came a cross Matrix Factorization for classification or recommendations. I was very shocked that it was one of the most effective techniques in the Netflix competition. But actually I think ...
1
vote
0answers
35 views

A new piece of clue for document classification?

I am working on a document classification problem. I am using the typical vector space model to represent a document as doc-term vector. If document has some term, the vector entry for that term is ...
1
vote
0answers
21 views

Compare averaged GLM with boosted regression trees using cross validation : d2 and RMSE calculation

I want to compare BRT and averaged glm models on test sets by calculating the explained deviance and RMSE. How can I calculate d2 and RMSE from predictions? I use the following functions: gbm1 ...
2
votes
0answers
39 views

Conceptual issues in training neural network and learning curve

I have a 4 Input and 3 Output Neural network trained by particle swarm optimization (PSO) with Mean square error (MSE) as the fitness function using the IRIS Database provided by MATLAB. The fitness ...
0
votes
0answers
22 views

Response surface of a particular discontinuous function

I have a function IR that depends on several (maybe 100) input random variables. I know ...
12
votes
2answers
3k views

What are the main differences between K-means and K-nearest neighbours?

I know that k-means is unsupervised and is used for clustering etc and that k-NN is supervised. But I wanted to know concrete differences between the two?
0
votes
0answers
34 views

Precision and recall Break-Even performance

I am using WEKA to do classification. I wish to compare with an existing paper that uses the Precision and Recall Break-Even performance as their results. Can someone please help in order for me to ...