Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms (1)

0
votes
1answer
44 views

Feature selection in GBM

I am using gradient boosting (caret package in R). As far as I understand, the feature selection is already included in this package. However, I slightly ...
0
votes
1answer
48 views

Using Adaptive Linear Neurons (Adalines) and Perceptrons for 0-1 class problems

I am wondering how to adjust the Adaline algorithm to classify the classes 0 and 1 instead of -1 and 1. I found a section in Neural Networks and Statistical Learning by Ke-Lin Du, M. N. S. Swamy ...
0
votes
0answers
32 views

Observation Likelihood in hidden Markov models

As far as I understand, in discrete HMM, the observation symbol probability distribution $b_{i}(O_{t})$ is always a probability less than 1, e.g. $\frac{1}{6}$ for each side when rolling a dice. But ...
2
votes
2answers
116 views

Ax = b. How can I estimate A, given multiple data vectors of x and b?

I have a problem and I believe there must be a machine learning technique to solve it, but I am new to machine learning and I have no idea where to start. So, I have multiple multivariate parameter ...
0
votes
0answers
39 views

The accuracy measure of a classification process

I have build the signal processing and feature extraction models, those features, inputed to Neural Network using matlab, which is give me the following performance measure,And I have several ...
1
vote
0answers
45 views

Regularized regression with missing data?

Are you aware of any regularized regression methods (i.e. Lasso, elastic net) which allows for using cases with incomplete (missing) data (e.g. using EM estimation)? And if yes, is the method ...
0
votes
0answers
28 views

Good library for Switching Autoregressive Hidden Markov Model (SAR-HMM)

can you recommend a good library for SAR-HMM? (Switching Autoregressive Hidden Markov Model) Apparently only MATLAB can be recommended. ...
0
votes
0answers
12 views

converting discrete values to buckets to perform predictions

I have a set of continuous discrete values, which I would like to convert to a classification task. Say, my scores in an exam are anything between 0-100. I want to convert my scores in the next exam ...
1
vote
1answer
142 views

Do I need to be a mathematician to be more than an expert in machine learning? [closed]

Machine learning has its roots principally in mathematics so if I wanted to be not only an expert but an innovator in this area would I have to be a computer scientist or would I need to be a ...
1
vote
1answer
49 views

Random forest regression on a given interval

I'm training a random forest regression model on a dataset that consinsts of values in the range of 0-50. It has many values close to zero and only 500 observations. The R^2 is also small, about ...
2
votes
1answer
170 views

A framework for multi-valued categorical attributes

In the scenario in which I'm working each entity could be represented in terms of N distinct properties that I will call p1, p2, ..., pn. For each of them, an entity, can have its specific range of ...
0
votes
1answer
19 views

difference between stochastic gradient descent and batch

I've watched the canonical Andrew Ng video on the subject but I'm trying to translate those concepts into java, and I'm not quite sure I did it right. The thing that's confusing me is, I made a toy ...
0
votes
0answers
22 views

Bayesian Ridge vs Stochastic Gradient Descent

I was running some Regression algorithms on a dataset and it just so happens, that the Bayesian ridge Regression techniques is performing not so well as the SGD (Stochastic Gradient Descent) ...
0
votes
0answers
36 views

Implementing WARP Loss (Gradient Computation)

I am trying to implement the WARP Loss in Torch, as defined in the WSABIE paper: http://www.thespermwhale.com/jaseweston/papers/wsabie-ijcai.pdf The Algorithm is as follows: The Algorithm ...
2
votes
2answers
63 views

Performance in training set worse than in test set?

I have a high dimensional regression problem where I used glmnet to solve this. A nested CV scheme is used. In the inner CV loop (10x5 fold) a grid search is done to find the optimal hyperparameters ...
0
votes
0answers
14 views

Query re. how to set up an SVM, which SVM variation … and how to define a metric

I’d like to learn how best set up a Support Vector Machine for my particular problem (or if indeed there is a more appropriate algorithm). My goal is to receive a weighting of how well an input set ...
0
votes
0answers
21 views

Number of hidden units in Restricted Boltzmann Machine?

In section 12.1 of Geoff Hinton's Practical Guide to Training RBM on how to choose the number of hidden units it is stated that one should ...
1
vote
1answer
26 views

Why does the equation $ -\sum^{n}_{t=1} \tilde{W}(t)_{m-1} y_{t}h(x; \theta_{m}) = 2 \epsilon_m -1$ hold in boosting?

I was trying to understand the boosting algorithm as described by the MIT graduate class lectures notes on ocw. On page 2 they give the outline of boosting as follows: The step that is not clear ...
6
votes
1answer
115 views

How do you mathematically prove that boosting cannot have zero error in training set arranged in a square with the corners labeled plus and minus?

I have the following data: Say we want to perfectly separate those points using only an ensemble of horizontal and vertical decision stumps. Maybe using Boosting or Adaboost, but the main point is ...
0
votes
0answers
35 views

Decomposing the non-deterministic transition functions in non-Markov decision processes into several deterministic transition functions

Problems in reinforcement learning are commonly modeled as Markov decision processes (MDPs). One essential part of MDPs is the transition function $T: S \times A \times S \rightarrow [0, 1] \in ...
1
vote
1answer
21 views

Adding additional data to a model

I'm currently running a logistic regression for classification on 100 patients dataset. However for 10 of those patients I have additional data that would help the accuracy for for those 10 patients. ...
0
votes
0answers
23 views

Hyper parameters optimization

Any one with a tip on how to define a suitable condition to find an optimal set of parameters for a combined norm regularization on a grid? Not cross validation or any related method. I am asking if ...
1
vote
1answer
71 views

Boosted trees and Variable Interactions

How can one see in a Boosted trees classification model, which variables interact with each other and how much? I would like to make use o this in R gbm package if possible
0
votes
1answer
26 views

What are the good study materials on Association Rules?

I am looking to learn Association Rules, from basic level. I was looking for some good web based materials to start with. My objectives in the materials is to: (a) learn the aspect nicely from ...
1
vote
1answer
28 views

Practical collaborative filtering application for large database

I’m designing an item-based collaborative filtering for a large database with over 100,000 items. My question is how the whole process works in practice since the algorithm takes a long time to ...
0
votes
2answers
97 views

Introductory multivariate statistics reference for beginners

I am from computer science department doing research in data mining and image mining. I remember the last course about stat was introductory to statistics and probability in general. Now I have this ...
1
vote
1answer
48 views

Non linearity in data using regression

I have been working with a set of data which is set in an engineering discipline. However my aim is predictive in nature, i.e., I need to get the relationship between the parameters as well as predict ...
0
votes
0answers
17 views

How to draw ROC curve for softmax classifier?

As softmax classifer is a generalized form of logistic regression, how to draw ROC curve for softmax classifier?
13
votes
1answer
430 views

What are the theoretical guarantees of bagging

I've (approximately) heard that: bagging is a technique to reduce the variance of an predictor/estimator/learning algorithm. However, I have never seen a formal mathematical proof of this ...
0
votes
0answers
34 views

Value of the loss function and Classification Errors in gbm package (R)

I have a simple problem of classification (0s and 1s) using adaboost loss function. When I check the components of a boosted model using the gbm package I see: ...
0
votes
0answers
4 views

Lower bound for the randomized perceptron algorithm

The above exercise is taken from the lecture notes:http://goo.gl/tlkpKc It is a kind of lower bound for the perceptron linear classification algorithm. I was able to show a sequence ...
4
votes
1answer
117 views

The difference between logistic regression and support vector machines?

I know that logistic regression finds a hyperplane that separates the training samples. I also know that Support vector machines finds the hyperplane with the maximum margin. My question: is the ...
1
vote
2answers
79 views

Why don't we train neural networks to maximize linear correlation instead of error?

Recently a project I've been a part of has involved training neural networks so that we maximize the Pearson correlation between actual and predicted values. So this came to my mind: why don't we ...
0
votes
0answers
23 views

Learning Curve Meaning

I have made a learning curve that looks like this: Why wouldn't it be more like both training and cross-validation score begin low and both gradually increase with more samples? Why does one start ...
1
vote
0answers
24 views

Standardizing dimension reduction output

I understand that data is (typically) standardized (i.e. zero mean and unit variance) before dimension reduction technique such as PCA/LDA is applied. In addition to this, would it ever make sense to ...
0
votes
0answers
21 views

Good baseline algorithm for text-related machine learning project

I'm working on a machine learning project aimed at predicting the quality/helpfulness of a review. For each review in the dataset, I have the review text, a number 'm' for the number of people who ...
0
votes
1answer
75 views

How large should the sample be for stochastic gradient descent?

I understand that stochastic gradient descent may be used to optimize a neural network using backpropagation by updating each iteration with a different sample of the training dataset. How large ...
14
votes
2answers
418 views

Why does gap statistic for k-means suggest one cluster, even though there are obviously two of them?

I am using K-means to cluster my data and was looking for a way to suggest an "optimal" cluster number. Gap statistics seems to be a common way to find a good cluster number. For some reason it ...
0
votes
1answer
89 views

How may I evaluate my self study of Machine Learning?

I am trying to learn Machine Learning on my own. To do this, I try to read from textbooks, good web-based materials like Stanford or Caltech. I try to work out my exercises, and try to discuss my ...
1
vote
1answer
39 views

What are the differences between filters learned in autoencoder and convolutional neural network?

In CNN, we will learn filters to produce feature map in convolutional layer. In Autoencoder, each layer's single hidden unit can be considered as filter. What the difference between the filters ...
6
votes
1answer
288 views

Machine learning classifiers

I have been trying to find a good summary for the usage of popular classifiers, kind of like rules of thumb for when to use which classifier. For example, if there are lots of features, if there are ...
1
vote
0answers
12 views

Will the classification accuracy vary if we first classify based on a single variable and then use the rest?

Let's suppose I am doing classification and that I have 99 features and another feature that says if the person is male or female. I have two options viz to build one classifier using all the ...
0
votes
0answers
13 views

What makes a good test set in an unbalanced corpus?

So, I have a machine learning classification problem, where the classes are quite unbalanced in the initial annotated dataset. My idea is to use Bootstrap method to help the classifier get new ...
0
votes
0answers
12 views

Document classification with very few documents

I wanted to know if there is any method in state of art that deals with document classification methods with very few training samples in R. I have just 20 documents and need to classify them into 3 ...
1
vote
1answer
56 views

Can a neural network have an image as its output layer?

According to what I've read, the output layer of a neural network is going to either perform "classification" or "regression". In regression, a numerical value is chosen on a single output node, and ...
0
votes
2answers
64 views

TF-IDF versus Cosine Similarity in Document Search

I'm wondering if anyone can help me out or point out some resources to learn more about TF-IDF and document search. I'm trying to implement a basic document search and am trying to better understand ...
0
votes
1answer
51 views

Why do RNN's have a tendency to suffer from vanishing/exploding gradient?

The title says it all: Why do RNN's have a tendency to suffer from vanishing/exploding gradient? For what is a vanishing/exploding gradient see: ...
0
votes
0answers
18 views

When does weight sharing in RNN's make sense (or not)?

It is my understanding that RNN's share weights. It seems to me that this may not be wise for all situations. So if you use an RNN (with weight-sharing) what are you assuming about the problem you are ...
1
vote
0answers
32 views

Bias and variance of a naive bayes classifier and KNN classifier

After reading the paper by J. Friedman, ”On bias, variance, 0/1-loss, and the curse-of-dimensionality,” Data Mining and Knowl- edge Discovery, 1997. I would like to estimate both bias and variance ...
4
votes
0answers
23 views

VC dimension of regression models

In the lecture series Learning from Data, the professor mentions that the VC dimension measures the model complexity on how many points a given model can shatter. So this works perfectly well for ...