Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms

1
vote
0answers
36 views

Squared Error vs Absolute Error loss functions [duplicate]

The two most popular types of loss functions are 1) squared error: $L(y,f(x))=(y-f(x))^2$ --> best estimate is the $E(Y|x) $ 2) absolute error: $L(y,f(x))=|y-f(x)|$ --> best estimate is the ...
0
votes
1answer
59 views

How to compare models from different but related datasets?

I'm building regression models on four the different but related data set and at the end, I want to test the significance of models. Since my models are built in a different data set, it's not ...
2
votes
0answers
54 views

Mixture of probits: understanding truncated-based likelihoods

I am trying to implement a mixture model of probits to infer the best decision boundary for every latent subpopulation. When doing Gibbs sampling, we eventually have to compute $P(y^* | w_c)$ where ...
1
vote
1answer
18 views

Regression Gaussian Estimates instead of points

I am using a support vector regression is order to get estimates of a variable y. I want to receive a probability distribution of my estimates and not just point estimates. I want to predict Gaussian ...
3
votes
1answer
50 views

When is it appropriate to use PCA as a preprocessing step?

I understand that PCA is used for dimensionality reduction to be able to plot datasets in 2D or 3D. But I have also seen people applying PCA as a preprocessing step in classification scenarios where ...
0
votes
0answers
18 views

After applying SVD, how do I find which features from my original dataset were most significant?

I'm using MathNet.Numerics library in C# to find the SVD but the Sigma matrix gives no indication of which values correspond to which features. It simply lists them in the most significant order. ...
0
votes
0answers
27 views

Calculating the variance of a model?

I often hear about the bias-variance tradeoff to evaluate classifiers. Now I want to calculate them. I often compute the AUC of a binary classifier to evaluate its performance and do a 10-fold ...
2
votes
0answers
48 views

Some doubt in reading Machine Learning A Probabilistic Perspective ( chapter 3.2 )

When I am reading Murphy's Machine Learning A Probabilistic Perspective. In chapter 3.2. I have some doubt. I think the author want to express is two things. First, we can use Bayes formula to ...
2
votes
1answer
92 views

What is the difference between a multi-label and a multi-class classification?

What is the difference between multi-label classification and multiclass classfication. Speficially, what is the difference between a label and a class? Please provide a clear example. "Multiclass ...
0
votes
0answers
27 views

Usefulness of Z-normalization in Machine Learning

Z-normalization means rescaling the feature $X$ by subtracting the average $\mu$ and dividing by its standard deviation $\sigma$, i.e., $(X-\mu)/\sigma$. What is the usefulness of normalizing data ...
1
vote
0answers
27 views

Binary classification of dated text documents with seasonality

I have a collection of training documents with publication dates, where each document is labeled as belonging (or not) to some topic T. I want to train a model that will predict for a new document ...
7
votes
3answers
384 views

How is cross validation different from data snooping?

I just finished "An Introduction to Statistical Learning". I wondered whether using cross-validation to find the best tuning parameters for various machine learning techniques is different from data ...
1
vote
1answer
92 views

Analysis of Customer satisfaction surveys

I have customer feedback data about 2-3 products from 100 customers. Number of questions are around 160. I have data in excel format. Header row contains the question and row below contains the ...
2
votes
0answers
71 views

Rademacher bounds for unbounded loss functions

All common treatment of PAC bounds based on Rademacher complexity assume a bounded loss function (for a self-contained treatemnt, see this handout by Schapire. However, I could not find any result for ...
3
votes
0answers
65 views

Duda, Hart, Stork No Free Lunch Discussion

Please see this question regarding Duda, Hart, and Stork's No Free Lunch Thm Discussion Hi all, I was having trouble understanding the description of the NFL theorem in Duda, Hart, and Stork. My ...
0
votes
1answer
42 views

Predicting the impact point of a moving object

Suppose we have a moving object (a horizontal projectile motion as one of the most basic examples). Is there any way to predict where it will hit finally? Please note that I'm looking for a machine ...
0
votes
0answers
21 views

How much time is reasonable for training Restricted Boltzmann Machine?

I have implemented Binary RBM in Matlab. I am using 60000 images as an input to train RBM. It takes approximately 11.3 minutes. I used tic and toc functions to evaluate above mentioned elapsed time. ...
3
votes
1answer
81 views

Variance-covariance matrix for ridge regression with stochastic $\lambda$

In ridge regression with design matrix $X$, outcomes $y$, fixed regularization parameter $\lambda$, and errors $\epsilon\sim\mathcal{N}(0, \sigma^2I)$, the computations for the ridge regression ...
0
votes
0answers
19 views

GB-RBM unable to learn and generate simple 2D-Motions?

I am trying to apply a GB-RBM to a variation of the LASA Handwriting Dataset to be able to generate new examples. My dataset contains motion trajectories of simple shapes like 'S' shapes, spirals, 'C' ...
1
vote
0answers
22 views

Bayesian Perceptron - how can I generate many different perceptrons?

I am going to implement the Bayesian version of a perceptron that I read in the Statistical Mechanics of learning, by Engel-Van Den Broeck. The idea to improve the performance is to use many Gibbs ...
0
votes
0answers
35 views

Conditional or Joint Probability under Various distributions

In various statistical models the baseline equation (like in Naive Bayes $$\mathrm{classify}(f_1,\dots,f_n) = \underset{c}{\operatorname{argmax}} \ p(C=c) \displaystyle\prod_{i=1}^n p(F_i=f_i\vert ...
1
vote
1answer
47 views

Where can I use kernels other than Gaussian (like Cauchy, laplacian) in kernel methods in machine learning? Or maybe in kernel density estimation?

In few papers I read that - kernel used doesn't really matter for kernel density estimation but bandwidth of the kernel is the most important factor. But I did not see any mathematical explanation to ...
0
votes
0answers
5 views

How accurate sum of kernel function needs to be, so that we can use it in Mean shift algorithm (may be for image segmentation)?

Mean shift is a procedure for locating the maxima of a density function given discrete data sampled from that function. It is useful for detecting the modes of this density. This is an iterative ...
4
votes
2answers
163 views

Ensembles of Ensembles?

I like the idea of ensemble learners, especially Bagging, but I always wondered as why they are not the most powerful learners since they have a clean motivation. I don't have the answer to that ...
0
votes
0answers
19 views

How to interpret merits in Weka with ChiSquaredAttributeEval and SVMAttributeEval?

I want to interpret the goodness of attributes using feature selection with 10-fold cross validation. With ChiSquared I get something like this (deletet attributes with merrit was 0 in all folds): ...
0
votes
0answers
24 views

Predict feature combination with highest probability

I trained a Support Vector Machine with the caret package in R. My dataset looks the following: ...
0
votes
0answers
19 views

Regression line fit for linearly increasing data with manual reset

I've a linearly increasing time series dataset of a sensor with value ranges between 50 to 150 on which I've implemented a simple linear regression algorithm to fit a regression line, and I'm ...
0
votes
0answers
18 views

Improving sentence segmentation in NLTK

I have been looking into problem of sentence segmentation lately. I have been referring to NLTK's book for this purpose. I followed their procedure to segment sentences presented here: ...
0
votes
0answers
30 views

What is the simplest way to classify airplan manuvers?

Suppose we have declared four motion types for air-plane. If we represent each maneuver with a trajectory line, what is the best classification method to retrieve the trajectory pattern with a similar ...
1
vote
2answers
245 views

Estimate ARMA coefficients through ACF and PACF inspection

I know that this is probably a question that's been asked plenty of times, but i haven't seen an answer that's both accurate and simple. How do you estimate the appropriate forecast model for a time ...
1
vote
0answers
12 views

determing states in HMM with BIC

I'm fitting a HMM to time series, for each data set I use BIC results to select the optimum number of states. In that, the BIC number is lowest and thereby indicating this model with that number of ...
4
votes
1answer
106 views

Is KNN a discriminative learning algorithm?

It seems that KNN is a discriminative learning algorithm but I can't seem to find any online sources confirming this. Is KNN a discriminative learning algorithm?
0
votes
0answers
10 views

Turning MiniBatchKMeans into Fuzzy MiniBatchKMeans

I'm using Scikit-Learn, which has an implementation of MiniBatchKMeans. I'm very inexperienced with ML, so I'm wondering how (if ...
2
votes
3answers
117 views

Best way to turn a date into a numerical feature?

I have a fairly large dataset with a few fields containing time-related data. This data comes in various shapes and sizes, but most of it can be parsed and rephrased in more appropriate formats for ...
0
votes
1answer
45 views

SVM cost parameter

In a SVM with linear kernel, could you explain to me what exactly the C parameter is/represents? An example why it's important to select a good value for C would also be appreciated. Thank you.
1
vote
1answer
84 views

Maximum Entropy Model for classification, what to use as context & feature?

I'm building a Maximum Entropy Model to classify some text, based on paper "A Maximum Entropy Approach to Natural Language Processing" by Berger et.al. It's similar to POS tagging. Below is some ...
0
votes
0answers
43 views

What is this problem called in machine learning research?

Given 2 sets of entities from 2 different classes described by properties $f_1...f_n$, any 2 entities from the 2 different classes together have a score: ${\rm Score}(c_1e_1,c_2e_1) = y$. How ...
2
votes
1answer
89 views

The usage of data mining in pharmaceutical companies?

I know that data mining applications are being used in pharmaceutical companies, but my question is: what do they use them for? Sometimes I read: "drug discovery", but how? How is it used for drug ...
0
votes
0answers
10 views

how to create labeled term to document matrix for classification for TMG

I already asked the same question but not got any reply. Please any one help me. I have created term-to-document matrix for document collection. I want to classify them with TMG (Term to matrix ...
1
vote
0answers
21 views

What are some multivariate models with feature interactions

I have dependent variable matrix $Y_{i,j}$ and feature matrix $X_{i,k}$. My objective is to predict each element of the vector $[y_{i,0},...,y_{i,J}]$ by using new observations of the features, ...
0
votes
0answers
53 views

Training and testing on Unbalanced Data Set

I used SMOTE algorithm in R for class balancing. My data size has 13000 rows, I had 7% minority class in my sample now I used SMOTE( Synthetic Minority Oversampling Technique) for class balancing such ...
0
votes
1answer
29 views

how to handle (many) false positives in training dataset for logistic regression classifier

I want to train a logistic regression dataset. I have a quite big training data set ( >100 000) and have around 10 features I can train on. Half of my training data is negative training data and I ...
0
votes
0answers
10 views

Latent semantic classification

How can I create a training data set for document classification using LSA? I have created a term-to-document matrix and have class labels also. I don't know whether to add these class labels in a ...
0
votes
0answers
31 views

How to choose the proper supervised machine learning algorithm

I have following data: ultra-sound audio recording, and I have the raw data of those samples, and of course the Fourier Transform data for those samples. Each dataset is assigned a value (for example ...
1
vote
1answer
75 views

Example of how the log-sum-exp trick works in Naive Bayes

I have read about the log-sum-exp trick in many places (e.g. here, and here) but have never seen an example of how it is applied specifically to the Naive Bayes classifier (e.g. with discrete features ...
3
votes
1answer
44 views

What is the difference between a “learner” and “classifier” in supervised learning?

This question stems from Pedro Domingos' excellent paper "A Few Useful Things to Know About Machine Learning." The paper is extremely clear and well-written, but I still have a clarification question. ...
5
votes
1answer
173 views

Stochastic Programming with MCMC

I have just started learning about MCMC (using PyMC), and it seems to be a hammer that can be used to solve a large class of inference and optimization problems. While I understand that there are ...
0
votes
1answer
57 views

Way to train Hidden Markov Model in R with multiple sequences

i have multiple sequences for each of two states. I'd like to train a HMM with these to predict the state for unkown sequences. Here is an example for this problem: ...
0
votes
0answers
10 views

Prediction with not atomic features

I would like to use another type of data, not atomic data, as a feature for a prediction. Suppose I have a Table with those Features: - Column 1: Categorical - House - Column 2: Numerical - 23.22 - ...
2
votes
1answer
54 views

How to prepare interactions of categorical variables in scikit-learn?

What is the best way to prepare interactions of categorical features before fitting with scikit-learn? With statsmodels I could conveniently say in R-style ...