Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms (1)

2
votes
1answer
164 views

How to get started in data-relation algorithms and mathematics?

I am very interested in the concepts and discussions taking place here. However, I'm not entirely sure what this field of study is called or what the many branches of study are being discussed here ...
2
votes
2answers
722 views

Which kernel function for Watson Nadaraya classifier?

I am trying to implement a Watson Nadaraya classifier. There is one thing I didn't understand from the equation: $${F}(x)=\frac{\sum_{i=1}^n K_h(x-X_i) Y_i}{\sum_{i=1}^nK_h(x-X_i)}$$ What should I ...
2
votes
1answer
440 views

GMM with Bayes decision model

Given two classes of training data (A and B), I want to fit each class' distribution using a GMM with k components, and then use Bayes Decision Model for the classification. The first step was to ...
11
votes
3answers
4k views

Semi-supervised learning, active learning and deep learning for classification

Final edit with all resources updated: For a project, I am applying machine learning algorithms for classification. Challenge: Quite limited labeled data and much more unlabeled data. Goals: ...
3
votes
1answer
200 views

Benefits on kernel trick and one related question

My questions are: What are the benefits of kernel trick? Can anyone summarize? One thing I read about in one lecture note is , 'we never need to explicitly represent feature vectors'. I do not ...
13
votes
2answers
3k views

What does interaction depth mean in GBM?

I had a question on the interaction depth parameter in gbm in R. This may be a noob question, for which I apologize, but how does the parameter, which I believe denotes the number of terminal nodes in ...
9
votes
3answers
4k views

How to quickly select important variables from a very large dataset?

I have a dataset with about 2,000 binary variables/200,000 rows and I'm trying to predict a single binary dependent variable. My chief goal at this stage isn't getting accuracy of prediction, but ...
2
votes
1answer
221 views

Different optimal number of boosting iterations obtained from OOB and on test

If I'm using a machine learning model (e.g. boosted regression trees like gbm in R) on a dataset, what does it mean if there's a significant difference between the OOB estimated optimal # of ...
1
vote
1answer
74 views

Getting critics to recognize that two similar input patterns refer to the same output-performance relationship

The actor-critic model is used within temporal difference learning, which is a method within reinforcement learning, to optimize a process on a state-by-state basis by using the difference between ...
4
votes
2answers
556 views

Machine learning in web application?

There are a lot of outstanding machine learning/data mining standalone applications available in different languages like Java, Python, and others. However, I wonder, practically, in case of applying ...
2
votes
2answers
82 views
0
votes
2answers
285 views

Factors that affect variation in the data?

What is the practical way to identify the factors that create variation in a data of a dataset? What category does this question fall into? Are there a set of algorithms that can be used for this ...
2
votes
0answers
78 views

Paired multiarm bandit

I have a set of independent experiments with different distributions and I'm trying to determine which has the highest mean payoff. I would like to treat this as a multi-arm bandit problem, but the ...
0
votes
0answers
377 views

Problem with GBM model

Alright, hopefully third time is the charm. I'm basically trying to build a predictive model with R and gbm. For various reasons, I can't explicitly state exactly what I'm doing. Basically I have a ...
1
vote
2answers
501 views

Metric for probability based classification

I am doing a system for classifiying documents. The project demands the use of probability based output. So a sample will have a probability for belonging to each class. For now I use logistic ...
1
vote
2answers
945 views

How can I implement multiclass k-NN?

I want to implement k-NN to use in a multi-class dataset. I found "A k-Nearest Neighbor Based Algorithm for Multi-label Classification" but didn't get the algorithm. Do you know any clear explanation ...
5
votes
1answer
1k views

Best practices for measuring and avoiding overfitting?

I am developing automated trading systems for the stock market. The big challenge has been overfitting. Can your recommend some resources describing methods for measuring and avoiding overfitting? I ...
4
votes
2answers
850 views

Validation of clustering results

I have a data which contains several columns which I later reduced using a PCA algorithms to two different components. I then applied the k-means algorithms to the data. Now, how can I verify that my ...
2
votes
0answers
98 views

Predictive model for network data

Assume a network as a set of data, which are defined by their coordination $(x,y,z)$ and a weight on its edge. Now this data can be used as an input data to predict a single value. In my case, ...
3
votes
3answers
584 views

Automatic text quality grading

I came up with an idea to use machine learning for automatic grading of topic-specific texts. More specifically, I will first use normal text classification techniques to sort all candidate texts ...
1
vote
0answers
70 views

How to characterize a problem of standardizing product descriptions

I'm looking for some advice for where to start on this problem. Let's say I have sales transaction data from a number of different retailers that all sell the same products. Even though they are ...
4
votes
2answers
474 views

Machine learning task with feedback loop

What are the available options if I want to perform a scoring task on a set of observations that: a) have a set of variables connected to them and, b) each round I get new information about the ...
1
vote
1answer
1k views

In natural language parsing, what is the feature function?

I'm working in the area of natural language processing, to be specific I'm reviewing the parsers that take advantage of data-mining techniques. I've read an introduction to natural language ...
3
votes
3answers
671 views

Text categorization/classification for small scale text

I'm looking into a way to classify/categorize sentences into pre-defined categories (around 10-15). Yes, indeed sentences, not articles or paragraphs. Given the average length of articles are not too ...
6
votes
1answer
366 views

Using a histogram to estimate class label densities in a tree learner

In a sequential (on-line) tree algorithm, I'm trying to estimate class label densities using a histogram. The algorithm grows a tree by generating test functions at each decision node, which ...
2
votes
1answer
176 views

How can it be proved that if number of samples are less than $d+1$, then the sample set is linearly separable?

Pattern classification by Duda, Hart, Stork (Section- 9.6.8) states that a 2-class training set of $d+1$ or less samples in a d−dimensional space is always linearly separable i.e. if the samples span ...
1
vote
0answers
114 views

Suggest an exhaustive procedure that will find a separating vector for linearly separable pattern in a finite number of steps

I read in a pattern classification text, that if we consider weight vectors whose components are integer valued, the perceptron procedure would terminate in a finite number of steps. What is the ...
8
votes
2answers
3k views

Interpreting distance from hyperplane in SVM

I have a few doubts in understanding SVMs intuitively. Assume we have trained a SVM model for classification using some standard tool like SVMLight or LibSVM. When we use this model for prediction ...
14
votes
5answers
5k views

Large scale text classification

I am looking to do classification on my text data. I have 300 classes, 200 training documents per class (so ...
2
votes
0answers
819 views

Ripper algorithm on large data sets

I've got a supervised learning problem where I've got about 15 features, mostly numeric, and I'm mapping these to a set of 5-7classes. Using decision trees or forests, I'm able to get confusion ...
2
votes
3answers
525 views

Learning multiple output

Can you suggest me an algorithm and probably a real code for multiple output learning, where input of the model is vector of around 10 000 values and output is, for each input vector, an output vector ...
-1
votes
0answers
83 views

R language-statistics-significance testing [duplicate]

Possible Duplicate: R language and statistics hypergeometric test This is in reference to my first question, although I have got an answer for it from Spacedman. But I am not entirely ...
0
votes
0answers
170 views

What is the appropriate method to use to calculate customer lifetime value?

I'd like to figure out what the potential lifetime value of a customer may be based on their purchasing patterns with our products. I have transactional data that tells me what a customer purchased, ...
5
votes
2answers
200 views

Project management for remote collaboration in prediction

Are there any tools for remote collaboration in prediction or machine learning settings? I am looking for a computing environment that includes appropriate source control, keeps track of how ...
1
vote
0answers
216 views

How does one prove that a separating hyper-plane exists for a linearly separable pattern?

How does one prove that a separating hyper-plane that can be represented as a linear combination of the training samples exists for a linearly separable pattern? Although, it looks pretty obvious ...
5
votes
3answers
234 views

Do image recognition efforts always rely on machine learning and statistics?

This is something I've always wondered. Consider the Kinect. It takes its 3d image data and manages to recognize that a human is contained at a given boundary. Are these types of technologies ...
10
votes
3answers
5k views

Support vector regression for multivariate time series prediction

Has anyone attempted time series prediction using support vector regression? I understand support vector machines and partially understand support vector regression, but I don't understand how they ...
1
vote
0answers
101 views

Bayesian classifier and discovery of new classifications

I've written Naive Bayesian classifiers before, they work wonderfully. But I'd like a classifier which will learn like a Bayesian classifier and identify new classifications when a new cluster ...
25
votes
3answers
4k views

How well does R scale to text classification tasks?

I am trying to get upto speed with R. I eventually want to use R libraries for doing text classification. I was just wondering what people's experiences are with regard to R's scalability when it ...
0
votes
0answers
217 views

What machine learning techniques are especially susceptible to “over-tuning” of their hyperparameters?

For example, the random forest algorithm is not especially susceptible to "over-tuning" because it has 1 hyperparameter, "mtry," and mtry typically does not have a large influence on the result of the ...
3
votes
2answers
147 views

What techniques are used for empirical, stochastic simulation of a time series?

Suppose you have recorded a set of paths in the $y,t$ plane, with $y = f(t)$, $f$ is a stochastic function (i.e. there is a noise term), and $t$ might be time or some other monotonic increasing ...
6
votes
1answer
429 views

Gibbs sampling for a simple linear model — need help with the likelihood function

So in order to better acquaint myself with Gibbs sampling, I've been working on a fairly simple linear model, written in Python/R. Basically, I have 2-dimensional input data (the xi) and a scalar ...
19
votes
4answers
512 views

To what extent is the distinction between correlation and causation relevant to Google?

Context A popular question on this site is " What are common statistical sins?". One of the sins mentioned is assuming that "correlation implies causation..." link Then, in the comments with 5 ...
4
votes
1answer
377 views

What is the $i$th sufficient statistic in the EM algorithm for Gaussian mixture models?

I am reading up on the EM algorithm for Gaussian Mixture Models, and there is consistent reference to the $i$th sufficient statistic. What is this, and why is it relevant to the algorithm?
9
votes
1answer
1k views

What is the best way to learn the fundamentals of probability required for machine learning algorithms?

I took a probability course in university a few years ago, but I'm going through some machine learning algorithms now and some of the math is just befuddling. Specifically right now, I'm learning ...
1
vote
1answer
76 views

Training sample division vs extra binary/nominal factor

Suppose, there is some classification/regression problem. It seems for me, that sometimes division of training sample by some feature (or maybe by some other reasonable method, say feature generation ...
1
vote
1answer
125 views

Labels in the Digit1 dataset for semi-supervised learning

I am working with the "Digit1" dataset introduced by the book "Semi-Supervised Learning" by chapalle et. al, as one of the benchmark datasets in the field. In the dataset description located at: ...
6
votes
5answers
3k views

Compare R-squared from two different Random Forest models

I'm using the randomForest package in R to develop a random forest model to try to explain a continuous outcome in a "wide" dataset with more predictors than samples. Specifically, I'm fitting one RF ...
1
vote
1answer
2k views

Text Classification in R [closed]

New to R, and am trying to do text classification. I am using R package tm to convert raw txt data into matrix. Here's the relevant code snippet. ...
1
vote
3answers
731 views

Software for drawing ROC curve

Having the sensitivity and specificity values, what software do you recommend that enables drawing the ...