Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms (1)

2
votes
2answers
682 views

Best machine learning algorithm for loans dataset?

I have a dataset of about 75K samples with about 20 features per sample (12 of which are probably important) describing various credit profiles - credit score, late payments, income, etc. Some of the ...
3
votes
2answers
786 views

Increasing the sample size does not help the classification performance

I am training a SVM classifier based on a given document collections. I started from using 500 documents for training, then I add another 500 for training, and so on. In other words, I have three ...
1
vote
1answer
96 views

Relationship between number of training set and classification performance

Are there any research/paper on the relationship between the number of documents for training and the classification performance using support vector machine?
0
votes
1answer
78 views

Defintion for model diversity?

Two models are diverse if they make prediction errors on different instances. I know there are different measures to quantify diversity, however, I'm looking for formal conceptual definition of what ...
6
votes
3answers
2k views

Importance of variables in logistic regression

I am probably dealing with a problem that has probably been solved a hundred times before, but I'm not sure where to find the answer. When using logistic regression, given many features $x_1,...,x_n$ ...
2
votes
1answer
164 views

Confusion related to Kalman filters density view

I was reading this book related to Kalman filters and I didn't understand a couple of things. I have also attached the screenshot of the pages from the book where I had confusion. The book is ...
1
vote
1answer
291 views

Coursera Neural Nets [closed]

If I go to https://www.coursera.org/course/neuralnets and click preview, there are many video lectures. Does anyone know if these are the same lectures you see if you enroll in a course?
8
votes
2answers
4k views

How to combine results of logistic regression and random forest?

I am new to machine learning. I applied logistic regression and random forest on a same dataset. So I get variable importance (absolute coefficient for logistic regression and variable importance for ...
9
votes
1answer
4k views

R vs Python for Data Analysis [duplicate]

Possible Duplicate: Python as a statistics workbench I am just starting out with data analysis and machine learning. From the books that I am reading/have read Python and R seem to be the ...
3
votes
1answer
6k views

Example of time series prediction using neural networks in R

Anyone's got a quick short educational example how to use neural networks (nnet in R for example) for the purpose of prediction? Here is an example, in R, of a ...
0
votes
1answer
154 views

Pointers for understanding the derivation of inference in linear dynamic systems

I am trying to learn about the inference and maximization basically EM of the linear dynamic systems(Kalman filters for example) from Bishop's book of Pattern Recognition and Machine Learning. ...
3
votes
1answer
192 views

Most relevant algorithms for Collaborative Filtering to test against

I am working on algorithms for collaborative filtering (CF). As part of this work, I want to compare a new algorithm to previous approaches to the problem. I am also surveying the most important ...
2
votes
2answers
568 views

Where does the definition of the hyperplane in a simple SVM come from?

I'm trying to figure out support vector machines using this resource. On page 2 it is stated that for linearly separable data the SVM problem is to select a hyperplane such that $\vec{x}_i\vec{w} + b ...
2
votes
0answers
143 views

Appropriate threshold to map a similarity value to an edge in a graph

In order to cluster users given a user-item binary matrix data, I am planning to first find user's similarity (Jaccard) and then use graph theory to isolate clusters (communities). I need to map the ...
0
votes
0answers
109 views

Can RBMs be used for feature selection / reduction?

I have a data set that's ~ 150R X 2000C and was curious if an RBM is appropriate in situation with this type of imbalance. It's a microarray and I'm looking at a 0/1 classification problem. I'd be ...
8
votes
2answers
627 views

Good measures of feature selection and class separability in classification machine learning problems

An example of a good measure of class separability in linear discriminant learners is Fisher's linear discriminant ratio. Are there other useful metrics to determine if feature sets provide good class ...
7
votes
1answer
310 views

Confusion related to linear dynamic systems

I was reading this book Pattern Recognition and Machine Learning by Bishop. I had a confusion related to a derivation of the linear dynamical system. In LDS we assume the latent variables to be ...
1
vote
0answers
753 views

Validation error less than training error — implications?

I am running a neural net to predict used car prices, sample size is 800. Using both 10-fold cross validation (10 times) and 1/3 holdback (10 times), the $R^2$ for training is about 0.60 and for ...
3
votes
3answers
509 views

Machine learning algorithm for ranking

I have got a set of elements $X$ which I can describe according to $n$ characteristics. Thus: $$x_i: \{c_{i1}, c_{i2}, \ldots, c_{in}\} \mid x_i \in X $$ where $c_{ij}$ is the (numerical) evaluation ...
2
votes
0answers
158 views

Confusion related to scaling factors in HMM

I was reading about HMM in C.M. Bishop's book Pattern Recognition and Machine Learning. I was going through the forward and backward algorithm using $\alpha$ & $\beta$ For forward messaging ...
1
vote
2answers
172 views

How to generate ROC Plot for semi-supervised algorithm?

By having a data-set 1000 (900 unlabeled, 100 labeled) record data-set for binary classification, I want to apply a semi supervised algorithm. The problem is that I don't know how to get values for ...
3
votes
0answers
176 views

What are the most popular domain adaptation methods (for transfer learning)?

I understand supervised and unsupervised learning well, and would be able to identify some 'basic' examples of, for example, supervised classifcation as: SVMs Random Forests Logistic Regression ...
6
votes
1answer
2k views

Distant supervision: supervised, semi-supervised, or both?

"Distant supervision" is a learning scheme in which a classifier is learned given a weakly labeled training set (training data is labeled automatically based on heuristics / rules). I think that both ...
2
votes
0answers
153 views

Random sampling for estimating mutual information - Time complexity and sampling error?

I have a dataset and I want to compute the mutual information (MI) for a selected set of variables. The dataset is large enough so that computation of the MI may take undesirably long time. Can I just ...
0
votes
0answers
77 views

How many parameters does a HM-SVM require?

How many parameters does a Hidden Markov Support Vector Machine require?
5
votes
2answers
247 views

Dependent Bernoulli trials

The probability of a sequence of n independent Bernoulli trials can be easily expressed as $$p(x_1,...,x_n|p_1,...,p_n)=\prod_{i=1}^np_i^{x_i}(1-p_i)^{1-x_i}$$ but what if the trials are not ...
0
votes
0answers
25 views

Dependent Bernoulli trials [duplicate]

Possible Duplicate: Dependent Bernoulli trials The probability of a sequence of n independent Bernoulli trials can be easily expressed as ...
3
votes
2answers
442 views

What is the computational complexity of the EM algorithm?

In general, and more specifically for Bernoulli mixture model (aka Latent Class Analysis).
4
votes
1answer
203 views

Machine learning predicted value

When we fit a generalized linear regression (e.g., logistic regression, gamma regression) we are estimating the population average Y given the predictors $X$ ( i.e., $E(Y | X)$ ). When we fit a ...
2
votes
1answer
363 views

In non-negative matrix factorization, are the coefficients of features comparable?

I'm using Alternating Nonnegative Least Squares Matrix Factorization Using Projected Gradient. The result (I use 2 as rank) is like this: ...
3
votes
0answers
434 views

Kernel SVM in primal training with Stochastic Gradient Descent

In short: I am currently reading Online Learning with Kernels (http://books.nips.cc/papers/files/nips14/AA33.pdf) for fun and I can't figure out how he got to equation 8 from equations 6 and 7. The ...
3
votes
1answer
5k views

What is “feature space”?

What is the definition of "feature space"? For example, When reading about SVMs, I read about "mapping to feature space". When reading about CART, I read about "partitioning to feature space". I ...
5
votes
1answer
3k views

Meaning of output terms in gbm package?

I am using gbm package for classification. As expected, the results is good. But I am trying to understand the output of the classifier. There are five terms in output. ...
4
votes
4answers
3k views

Machine learning book with code examples

I am studying Machine Learning and implementing ML algorithms with Matlab. I follow Ethem Alpaydin, Duda and Hart, Bishop and Mitchell's books. However none of them exactly give the pseudo codes and ...
4
votes
2answers
957 views

Determine the optimum learning rate for gradient descent in linear regression

How can one determine the optimum learning rate for gradient descent? I'm thinking that I could automatically adjust it if the cost function returns a greater value than in the previous iteration (the ...
2
votes
0answers
132 views

Statistical comparisons of multiple classifiers performance?

If the accuracy of $classifier1$ is statistically significantly better than $classifier2$ as per some hypothesis test, and likewise the accuracy of $classifier2$ is statistically significantly better ...
3
votes
1answer
1k views

Looking for examples or alternatives to R RuleFit ensemble package

Does anyone know of any good example code illustrations for the rulefit Rule Based Learning Ensembles package? The documentation is incredibly lacking. I was guided to the package by this paper. If ...
2
votes
0answers
276 views

Anomaly detection in user behaviour using hidden Markov models

I would like to detect user anomalies or mal-behavior on a web site. For each user I monitor the web browser used, IP (and thus ISP & geo-location) of the user as well as users' activities on the ...
3
votes
1answer
158 views

SVM optimization problem

I think I understand the main idea in support vector machines. Let us assume that we have two linear separable classes and want to apply SVMs. What SVM is doing is that it searches a hyperplane ...
8
votes
2answers
4k views

How to statistically compare the performance of machine learning classifiers?

Based on estimated classification accuracy, I want to test whether one classifier is statistically better on a base set than another classifier . For each classifier, I select a training and testing ...
1
vote
0answers
208 views

Matrix factorization vs random walk with restart for recommender systems

Suppose I want to handle "friend recommendation" problem on a large social network graph. I came across random-walk-with-restart as one technique used. I was thinking of using matrix factorization as ...
2
votes
1answer
196 views

Learn a joint distribution from incomplete samples

Suppose I want to learn a joint distribution $p(x_1, \ldots, x_n)$ and have a collection of samples $x^k_1, \ldots, x^k_n$ for each $k$. Assume some values $x^k_i$ are unknown, so the samples are ...
6
votes
1answer
394 views

Variational inference for nested Chinese restaurant process

I recently read paper by Chong Wang and David M. Blei "Variational Inference for the Nested Chinese Restaurant Process". And I couldn't understand the next part (from p.5): The variational update ...
3
votes
0answers
95 views

Data Mining / ML applications in hydrodynamics?

I have a question about Scientific Data Mining. Do you know successful case studies of applying Data Mining / Machine Learning techniques in hydrodynamics? In general, does it make actually sense to ...
4
votes
2answers
760 views

How can I transform time series data so I can use simpler techniques for fault prediction?

I know this is primarily a statistics site, so if I am off-topic, please redirect me. I have a system with pumps that sometimes break and need to be replaced. I would like to be able to predict the ...
3
votes
0answers
137 views

Using taxonomic levels as factors in random forests: does it make sense? Is it needed?

I want to test the effect of a set of predictors (ecological and morphological factors) on a categorical response variable (an animal behaviour). As far as I've read, random forests do not make ...
1
vote
1answer
860 views

Algorithm for price optimization [closed]

I'm trying to figure out a way for calculating price optimization in a commerce environment. In other words, I'm trying to analyze how a company can increase revenue and profitability by analyzing ...
2
votes
1answer
180 views

Is it necessary for a distance measure used in clustering to correspond to some valid vector space?

I have defined an distance measure based on some properties of points. But I'm not even sure that it corresponds to a valid distance in some vector space. Is this a necessary condition for clustering ...
1
vote
0answers
266 views

Univariate feature ranking in classification

Scikit-learn has function to evaluate the F-statistics for univariate feature importance feature selection. According to the web page they are calculating ANOVA F value. If I understood correctly, ...
1
vote
0answers
316 views

Stochastic Diagonal Levenberg Marquardt in Convolutional Neural Networks

Can someone explain this method of optimized convergence used in CNNs? I understand it involves generating the hessian matrix for every epoch, but can someone outline the steps? Thanks!