Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms (1)

0
votes
0answers
25 views

What is the definition of generalization error and its justification?

I was trying to understand rigorously what the goal of machine learning is. One could frame that one of the central goals of machine learning is to obtain the best possible function ever. But what ...
1
vote
0answers
17 views

measure the quality of feature vectors in terms of AUC and training time

I have computed 10 features for each example. I did an experiment to figure out which combinations of features can give the best classifier performance. I'm using libSVM for training. I noticed if I ...
1
vote
1answer
26 views

The meaning of the output from grid.py in libsvm

I'm a newbie in SVM, and have several questions regarding a tool in libsvm. There's tools/grid.py which tools/README explains as "parameter selection tool for C-SVM classification using 47 the RBF ...
1
vote
0answers
24 views

Why does value iteration converge to the optimal value function in **finite** amount of steps?

Recall reinforcement learning at (specially for notation): http://cs229.stanford.edu/notes/cs229-notes12.pdf Assume that we have a finite MDP. Why is it that value iteration converges to the optimal ...
3
votes
1answer
44 views

How to compute the R-squared for a transformed response variable?

Suppose that I run a linear regression y = x*b + error, and obtain predictions y_p. Furthermore, assume that I can compute the R-squared by calling a function R(y, y_p) which has two arguments: a ...
7
votes
3answers
164 views

Why aren't power or log transformations taught much in machine learning?

Machine learning (ML) uses linear and logistic regression techniques heavily. It also relies on feature engineering techniques (feature transform, ...
0
votes
0answers
10 views

Are these three different ways of expressing the optimal value function $V^*$ the same? (reinforcement learning)

My question didn't really fit on the title but its the following are the three following equations actually the same: $$V^*(s) = \underset{\pi}{max}V^{\pi}(s)$$ and $$V^*(s)=R(s)+ \underset{a \in ...
1
vote
1answer
23 views

Why is the optimal policy in reinforcement learning, independent of the initial state?

I was following the reinforcement learning lecture notes on CS229 (which can be referenced for the notation I am using in this question): http://cs229.stanford.edu/notes/cs229-notes12.pdf and I had ...
0
votes
0answers
22 views

What is a kernel parameter in Extreme Learning Machines?

I am using the MATLAB function elm_kernel but I don't know what the Kernel_Para variable do. Where can I learn about this? Also Regularization_coefficient. It looks similar to ...
1
vote
1answer
43 views

How does machine learning handle the vast number of words in NLP application

Disclaimer: I'm no expert in the field, this question could be axiomatically flawed. When programming a very basic regression based machine learning algorithm one uses a number of variables; it seems ...
0
votes
0answers
115 views

What distribution is the expectation taken over in the total expected pay-off in reinforcement learning? Is it consistent with Bellman's Equation?

I was following the reinforcement learning lecture notes on CS229: http://cs229.stanford.edu/notes/cs229-notes12.pdf on page 3 they have the equation for the expectation of the total pay-off: $$ ...
0
votes
0answers
19 views

adaboost with multiple classification algorithms

Up to now I saw that all adaboost implementations use single classification algorithm and a training dataset as input and then creates multiple classification models by re-sampling dataset and uses ...
0
votes
0answers
21 views

Normalization of Naive Bayes output

In Scikit-learn documentation it is possible to see that the MultinomialNB estimator has a method called predict-proba in which it has the following description: "Returns the probability of the ...
2
votes
1answer
60 views

how to differentiate matrix

How can I differentiate the following by $\mathbf{W}$ ? \begin{equation} \mathbf{Y} = (\mathbf{W}^T\mathbf{x} + b)^2 \end{equation} Where $\mathbf{W} \in \mathcal{R}^{d\times D}$ and ...
0
votes
0answers
82 views

Best machine learning methtod for classificating datasets with non-independent cases within the groups

I have to perform binary classification of my data with supervised machine learning, but I have some difficulties working with my data set. It consists many genetic mutations that have parameters ...
3
votes
1answer
50 views

Should I ever manually modify training data?

TL;DR: I'm reviewing a computer vision + machine learning module that someone else wrote, and I've discovered that she is manually cleaning up training data. Is that ever a good idea? The Details ...
0
votes
0answers
5 views

testing assuming a random design versus a fixed design

Why is hypothesis testing in, say linear models, performed while conditioning on the design. This in contract to the machine learning literature, where the risk is typically computed assuming a random ...
1
vote
1answer
39 views

AdaBoost over blackbox weak classifier

Can I somehow implement AdaBoost procedure over a weak classifier from another library? For example over SVM from libsvm, or over some neural network. The idea of AdaBoost is that current weights of ...
0
votes
1answer
33 views

Gibbs Sampling for Boltzmann Machines

David Mac Kay, in his book on machine learning talks about Boltzmann machines, and on pg. 3 here http://www.inference.phy.cam.ac.uk/itprnn/ps/521.526.pdf He says "the second equation ...
3
votes
1answer
48 views

maximum a posteriori vs squared loss

I am unclear about max a posteriori and squared loss. Let me assume I have $N$ images and $\mathbf{y}_i$ is the label of the image $i$, where, $\mathbf{y}_i\in \mathbb{R}^{C\times 1}$ - a binary ...
0
votes
0answers
63 views

Support vector machine questions

I have a question about the book by Bishop in chapter 7 pages 327-328 (pdf: http://www.rmki.kfki.hu/~banmi!/elte/Bishop%20-%20Pattern%20Recognition%20and%20Machine%20Learning.pdf). I don't understand ...
0
votes
0answers
26 views

How are ELMs tuning-free?

I have read that Extreme Learning Machines do not need any kind of iterative parameter tuning. However, in the MATLAB implementation of ELM that I use, I have a variable ...
1
vote
0answers
21 views

A paper that proves using the latent features of RBM as input to logistic regression?

I'm looking for a paper that includes a proof that simply training a Restricted Boltzmann Machine and then using the latent features as input to a logistic regression classifier is a correct thing and ...
0
votes
0answers
44 views

Is this a perfect classifier?

Let's say I have a multiclass single-label rule-based classifier and 1,000 retrieved documents which could be classified as: Class A -> 100 Class B -> 300 Class C -> 100 Class D -> 200 Class E -> ...
2
votes
0answers
44 views

Why does this multi-response Guassian LASSO not give a sparse solution?

I tried the glmnet package to learn multi-response Gaussian family. I have looked at the coefficients of the final model. The result is odd. All the features have ...
1
vote
1answer
74 views

Weka - Result interpretation

I am running the classify in Weka for a certain dataset and I've noticed that if I'm trying to predict a nominal value the output specifically shows the correctly and incorrectly predicted values. ...
1
vote
3answers
103 views

Class imbalance in Supervised Machine Learning

This is a question in general, not specific to any method or data set. How do we deal with a class imbalance problem in Supervised Machine learning where the number of 0 is around 90% and number of 1 ...
0
votes
2answers
57 views

How to determine precision and recall?

I am starting with Machine Learning and I have some trouble identifying true negatives and false positives from my classified data set. I have a classifier which classifies items in three classes. ...
2
votes
3answers
52 views

Find Neural Network Inputs Given Outputs

I've trained a neural network with two inputs, a single hidden layer with two neurons, and one output using a bipolar sigmoid activation function. If a single input is known, how would I determine the ...
1
vote
0answers
7 views

Separability measures

I am just studying pattern recognition. In that regard, my question is Why we need separability measures? Would you please give me detail explanation and suggest some books to read?
6
votes
7answers
780 views

Why not validate on the entire training set?

We have a dataset with 10,000 manually labeled instances, and a classifier that was trained on all of this data. The classifier was then evaluated on ALL of this data to obtain a 95% success rate. ...
5
votes
1answer
191 views

The Differences between Randomized Logistic Regression and Plain-Vanilla Logistic Regression

I would like to know the differences between Randomized Logistic Regression (RLR) and plain Logistic Regression (LR), therefore, I am reading a paper "Stability Selection" by Meinshausen, et al.; ...
2
votes
1answer
31 views

System identification, machine learning and time series

I have recently become a bit familiar with the machine learning techniques, and examples of problems where they are ought to be applied. For example, we can try deriving models for the time series or ...
17
votes
4answers
374 views

What makes the Gaussian kernel so magical for PCA, and also in general?

I was reading about kernel PCA (1, 2, 3) with Gaussian and polynomial kernels. How does the Gaussian kernel separate seemingly any sort of nonlinear data exceptionally well? Please give an intuitive ...
1
vote
0answers
31 views

Machine learning approach for modeling stochastic observations [closed]

Let X={xi, i=1:N}, where xi ranges within [a,b], and the distribution of xi is bimodal, but noisy. Now we have several observed instances of X: X’={xi’, i=1:N}, X’’={xi’’, i=1:N} etc, where {xi’} ...
0
votes
1answer
34 views

How to prove absolute lack of correlation

I have a huge dataset of 17 variables. I intended to use 15 of those to predict the 17th, and I could not find any model (ANN) to do so. I know that one of those variables definitely predicts the ...
0
votes
1answer
56 views

One class SVM to detect outliers

My problem is I want to build a one class SVM classifier to identify the nouns/aspects from test file. The training file has list of nouns. The test has list of words. This is what I've done: ...
0
votes
0answers
22 views

Does a Restricted Boltzmann Machine model any distribution as a Gibbs distribution?

What i know is that a Restricted Boltzmann Machine (RBM) is a Markov Random Field (MRF) and that the joint distribution of an MRF represents a Gibbs distribution. What I also know is that our goal of ...
0
votes
0answers
25 views

weka CVParameterSelection for libsvm(one class)

I'm trying to find out optimal parameters for one class libsvm. I read http://weka.wikispaces.com/Optimizing+parameters. I gave the values as G 0.01 0.1 10 ...
7
votes
1answer
413 views

Is the machine learning community abusing “conditioned on” and “parametrized by”?

Say, $X$ is dependent on $\alpha$. Rigorously speaking, if $X$ and $\alpha$ are both random variables, we could write $p(X\mid\alpha)$; however, if $X$ is a random variable and $\alpha$ is a ...
10
votes
5answers
367 views

Inference vs. estimation?

What are the differences between "inference" and "estimation" under the context of machine learning? As a newbie, I feel that we infer random variables and estimate the model parameters. Is my this ...
1
vote
0answers
25 views

bias term in online SGD [closed]

In this paper ...
4
votes
2answers
199 views

Autoencoders can't learn meaningful features

I have 50,000 images such as these two: They depict graphs of data. I wanted to extract features from these images so I used autoencoder code provided by Theano (deeplearning.net). The problem ...
7
votes
4answers
209 views

Are there any non-distance based clustering algorithms?

It seems that for K-means and other related algorithms, clustering is based off calculating distance between points. Is there one that works without it? Thanks!
0
votes
1answer
57 views

Which statistical melhods could I use to determine if a price is good, based on a history of prices?

I have the following scenario: A history of prices of a specific product; The current price of the same product. The history of prices should contain prices with a certain amount of discount, very ...
2
votes
0answers
14 views

False Positives in Real Time Classification

I am doing a sliding window binary classification. I have time series data and I am running a time window over this data and let a classifier produce a decision probability. Based on this probability, ...
0
votes
0answers
38 views

Combining multiple feature subsets through ensemble classification methods?

I have a set of $N$ samples to be classifies in a binary classification problem. I have extracted features from these samples from 4 different perspectives (views) of every samples. Hence I have 4 ...
0
votes
1answer
56 views

random forest modelling with high dimensional data

I am puzzling on developing random forest regression of high dimensional data. My predicted variable is plant cultivar or Class (say 1, 2, 3) and regresser are 82 variable in separate column (40 X 83) ...
2
votes
1answer
99 views

How do CNN's avoid the vanishing gradient problem

I have been reading alot about convoloutional neural networks and was wondering how they avoid the vanishing gradient problem. I know deep belief networks stack single level auto-encoders or other ...
1
vote
0answers
18 views

How to learn (arbitrary) constraints for selecting the best candidate from a group?

In my classification problem, each instance is a group of possibly hundreds of candidates, from which only one should receive the label $True$ and the remainder the label $False$. For example, in ...