Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms (1)

-2
votes
1answer
174 views

How could a t-test be used for comparing two ML algorithms?

I am new to ML, and I am reading a paper about ML comparison. It underlines the comparison between two algorithms based on the t-test but it does not describe the process further. I know what a ...
0
votes
0answers
120 views

Binary classifier probability measure?

I've the following situation. I've a binary classifier which classifies input feature vectors into either of two classes '$y$' or '$n$', along with the probability of it being in either of the ...
2
votes
1answer
78 views

Effect of varying outcome duration in longitudinal studies

I have a supervised classifier model (regularized discriminant) which predicts the probability of an event occurring within two years. This model was developed using sensor data measured from a ...
6
votes
2answers
3k views

Use of the Gamma parameter with support vector machines

When using libsvm, the parameter $\gamma$ is a parameter for the kernel function. Its default value is setup as $\frac{1}{Number Of Features}$ Is there any ...
3
votes
2answers
432 views

How to combine the responses of two sensors?

I have two sets of responses from two different sensors. In each set, the first column is distance measured in feet, and the second column is the response of the sensor. Sensor A has response values ...
10
votes
2answers
5k views

libsvm “reaching max number of iterations” warning and cross-validation

I'm using libsvm in C-SVC mode with a polynomial kernel of degree 2 and I'm required to train multiple SVMs. Each training set has 10 features and 5000 vectors. During training, I am getting this ...
1
vote
0answers
192 views

How to understand / visualize the error surface in online learning algorithms

I have a question about the shape of the error surface for online gradient descent algorithms. Take into account that I am trying to translate my specific question into a more general and idealized ...
0
votes
0answers
86 views

Confusion related to hidden Markov model

I am referring to this tutorial: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. I am a bit confused about the forward algorithm. From the tutorial: If we ...
0
votes
2answers
1k views

Learning Algorithms/Neural networks books? [duplicate]

Possible Duplicate: Machine learning cookbook / reference card / cheatsheet? Machine learning self-learning book? What are some good theoretical and practical machine learning algorithms ...
1
vote
0answers
34 views

Advice needed on optimal sensor selection and rules inference

I observe a certain system with a rectangular array of sensors, that can be either triggered or not. I know the system behavior can be well described by the sequence of triggering events and by the ...
9
votes
2answers
1k views

How to prove there is no finite-dimensional feature space for Gaussian RBF kernel?

How to prove that for the radial basis function $k(x, y) = \exp(-\frac{||x-y||^2)}{2\sigma^2})$ there is no finite-dimensional feature space $H$ such that for some $\Phi: \text{R}^n \to H$ we have ...
2
votes
1answer
794 views

Normalization of categorical factor variables

For supervised machine learning for prediction, if I had some feature variables that are real, and also some features that are categorical--which have been coded using dummy variables (010, 001 ...
3
votes
1answer
125 views

Smoothing a 2-by-2 contingency table

I am trying to implement a system for automatic document categorization, where each document of a corpus belongs to some class. I define the following contingency table for every class C and every ...
0
votes
0answers
112 views

Adding training examples to Bayesian classifier reduces accuracy

I'm working on a problem to predict/classify overall sentiment of a large amount of text, which I can verify on the next day. Each data point is a day and is composed of multiple articles. I bin the ...
7
votes
1answer
2k views

Why is pruning not needed for random forest trees?

Breiman says that the trees are grown with out pruning. Why? I mean to say that there must be a solid reason why the trees in random forest are not pruned. On the other hand it is considered very ...
3
votes
1answer
1k views

Trouble minimizing perplexity in LDA

I am running LDA from Mark Steyver's MATLAB Topic Modelling toolkit on a few Apache Java open source projects. I have taken care of stop word removal (for e.g. words such Apache, java keywords are ...
6
votes
2answers
249 views

What is the advantage of reducing dimensionality of predictors for the purposes of regression?

What are the applications or advantages of dimension reduction regression (DRR) or supervised dimensionality reduction (SDR) techniques over traditional regression techniques (without any ...
14
votes
2answers
2k views

How to get started with neural networks

I'm completely new to neural networks but highly interested in understanding them. However it's not easy at all to get started. Could anyone recommend a good book or any other kind of resource? Is ...
2
votes
2answers
466 views

Finding most informative feature subsets given dataset, clustering algorithm and gold standard partition

I have an $n \times m$ matrix of data $\mathbf{D}$ as well as a $k$-partition $P$ of $n$ indices each representing a row in $\mathbf{D}$. Assuming an arbitrary clustering algorithm $A$, I would like ...
21
votes
1answer
8k views

Does the optimal number of trees in a random forest depend on the number of predictors?

Can someone explain why we need a large number of trees in random forest when the number of predictors is large? How can we determine the optimal number of trees?
0
votes
1answer
187 views

R package for feature set algorithm selection

I want to train a binary classification NN and part of this will require data pre-processing. However, I have a choice of which pre-processing algorithm to use. Of course I'd like to choose that one ...
0
votes
1answer
120 views

Rebalancing discrete probability distribution

I'm working on a machine learning algorithm and have gotten stuck with how to rebalance a discrete probability distribution. I have a distribution represented as a simple array of $n$ numbers which ...
3
votes
3answers
2k views

Choice of neural net hidden activation function

I have read elsewhere that one's choice of hidden layer activation function in a NN should be based on one's need, i.e. if you need values in the range -1 to 1 use tanh and use sigmoid for the range 0 ...
3
votes
2answers
547 views

Bayes decision boundary of Figure 2.5 in Elements of Statistical Learning

When I read "Elements of Statistical Learning", I met some difficulty in calculating the Bayes decision boundary of Figure 2.5. In the package ElemStatLearn, it ...
0
votes
1answer
859 views

How to avoid multicolinearity in SVM input data?

Do you know of any techniques that allows one to avoid and get rid of multicolinearity in SVM input data? We all know that if multicolinearity exists, explanatory variables have a high degree of ...
2
votes
0answers
62 views

General rules for choosing machine-learning algorithm? [duplicate]

Possible Duplicate: Machine learning cookbook / reference card / cheatsheet? There are numerous machine-learning approaches out there. Also there are numerous ways how to optimize their ...
1
vote
0answers
399 views

Feature selection for SVM and Maximum Entropy

In text classification problems where the number of features >> number of documents, is it useful to perform feature selection with filters (e.g. Information Gain) when using Naive Bayes. However, ...
4
votes
1answer
1k views

Naive Bayes fails with a perfect predictor

Let's say I have a variable that perfectly predicts one of the classes in my dataset: ...
2
votes
1answer
767 views

What are the rules / guidelines for downsampling?

I have a data set with ~ 7 million rows, of which ~ 100k are positives. I'm looking to shrink the data by keeping all the ...
3
votes
0answers
340 views

Standardize/normalize power law distribution for machine learning

If my data follows a normal distribution I can standardize it for a machine learning algorithm, e.g. logistic regression, by subtracting the mean and dividing the result by the standard deviation. ...
3
votes
2answers
994 views

Why do I need bag composition to calculate OOB error of combined random forest model?

Could someone please explain me the answer of the question already given here: Combining randomForests in R, why are the err.rate, mse and rsq components NULL I do not understand the phrase "clip ...
1
vote
1answer
634 views

How to convert multiple ranking scores into a probability distribution?

I would like to create a topic distribution for a document. The current model I am trying to implement is: for each sentence in the document, I am getting a topic assignment with a score, e.g. "1st ...
1
vote
1answer
349 views

Cross validation accuracy is the same as the fraction of negative labels - what does it mean?

I have a dataset for classification (binary - 1/0) that has around 4000 samples that I use to train the model (I'm using an SVM, if that's relevant). To check whether everything is working fine, I ...
3
votes
1answer
213 views

On Elo++ updating rule

I am not sure if this is the correct place to ask this kind of question, I hope it is. I am studying this paper on an improvement of Elo rating system called Elo++. On page 4 the author states that he ...
3
votes
0answers
361 views

A good Machine Learning Book [duplicate]

Possible Duplicate: Machine learning self-learning book? I am a researcher on Machine Learning and I generally try to refer to Andrew Moore's Tutorials and Simon Haykin's Neural Networks. ...
1
vote
0answers
33 views

Evaluation and Testsets for NNMF

I am trying to evaluate my recommender system which uses Non-negative Matrix Factorization. Some things that I evaluate are How does the size of the feature matrix affect the recommendations How ...
2
votes
1answer
158 views

Representation within a RKHS framework

Given a p.s.d kernel $Q$, can minimization/maximization of $Tr(X^TQX)$ over X be represented within a reproducing kernel Hilbert space (RKHS) framework? If there is a primary concern with the trace ...
2
votes
0answers
370 views

Genetic algorithms, genetic programming or machine learning algorithms for solving this problem

I have a problem that consists of finding the optimal solution based on the following criteria: Logic for identifying that event A has occurred (i.e. "find" logic that most accurately categorises an ...
1
vote
1answer
253 views

Recommendations for MRI classification in R of large dataset (n=100, p=20000)

I am working on a magnetic-resonance imaging dataset which includes about 100 observations (= subjects) and 20000 predictors (=voxels). I would like to conduct classification in R using methods like ...
17
votes
3answers
2k views

Machine learning techniques for parsing strings?

I have a lot of address strings: 1600 Pennsylvania Ave, Washington, DC 20500 USA I want to parse them into their components: ...
6
votes
1answer
1k views

Common weak learners for Adaboost

I'm looking for a set of weak classifiers that work with Adaboost to test on popular datasets. Most of the examples on the web use some kind of random weak learners which work on their own randomly ...
1
vote
0answers
78 views

Distance correlation and prediction

If the distance correlation (ref. Gabor J. Szekely) $R_n(X,Y)>R_n(Z,Y)$ would the expected generalization error of a prediction model over $(Z,Y)$ be lower than $(X,Y)$ in predicting $Y$, where ...
10
votes
4answers
1k views

Appropriate clustering techniques for temporal data?

I have temporal data of activity frequencies. I want to identify clusters in the data that indicate distinct periods of time with similar activity levels. Ideally I want to identify the clusters ...
5
votes
0answers
511 views

Manifold regularization using laplacian graph in SVM

I'm trying implement Manifold Regularization in Support Vector Machines (SVMs) in Matlab. I'm following the instructions in the paper by Belkin et al.(2006), there's the equation in it: $f^{*} = ...
5
votes
4answers
826 views

How to handle online time series forecast?

I have been dealing with the following problem. I have kind of a real time system and every time frame I read its current value, creating a time series (such as 1, 12, 2, 3, 5, 9, 1, ...). I'd like to ...
2
votes
1answer
101 views

Is a different CV arrangement the same as a validation set?

I have a smallish dataset ~ 1500 rows X 500 columns. I've been using a standard 5 fold CV setup where row 1 = CV set1, row2 = CV set2, ... row 6 = CV set1,etc. I'm at the point where I'm trying to ...
1
vote
1answer
205 views

Possible reason for failing to build a support vector machine

I was trying to build a classifier for a set of documents using a support vector machine. I choose to build the feature space using term occurrence. While experimenting, I found the following ...
5
votes
1answer
184 views

Predictive Probabilities

Other than a calibration plot, is there a way to decide how good one models' predictive probabilities as compared to another model. I'm not interested in error rates as I find them ineffective for ...
2
votes
1answer
1k views

k-fold cross-validation strategy for large data set in statistical learning

I'm trying to learn the Bayesian network structure from a very large data set, and the R package I used for learning can only handle a very small portion of the data set (~10%) at one time due to the ...
1
vote
2answers
182 views

Computing the likelihood gradient on a simple directed graphical model with hidden unit

SHORT VERSION: We have a ('visible') random variable $X$ and a ('hidden') random variable $Z$. We have chosen appropriate distributions $P(X|Z)$ and $P(Z;w)$ where $w$ is the parameter of the model. ...