Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms (1)

1
vote
1answer
47 views

Method for solving problem with variable number of predictors (repost from Data Science)

REPOST from Data Science: I've been toying with this idea for a while. I think there is probably some method in the text mining literature, but I haven't come across anything just right... What is/...
3
votes
3answers
333 views

How to get a valid distance metric?

I have got a problem to devise a distance metric to get the similarity measurement of vectors. Someone suggested me to use dot product, which seems to me the same as the Cosine similarity metric; ...
3
votes
2answers
202 views

Full batch backpropagation implementation

I am trying to wrap my head around using batch backprop in a neural network. I have a very code-oriented mind, and I'm trying to figure out whether it's possible to parallelize the full batch ...
1
vote
1answer
617 views

Sparse coding vs. sparse PCA, are they the same thing?

Are they the same thing? If not, could someone possibly explain the difference or point to the seminal papers describing the approaches? I am looking not for a detailed technical exposition, but a ...
0
votes
1answer
106 views

What is the “standard reference” for cascade forward neural network?

What is the "standard reference" that firstly describes or surveys in details the cascade forward neural network? This kind of net is available in matlab toolbox for long cascadeforwardnet (as early ...
1
vote
0answers
104 views

serial correlation, homocedasticity tests for non linear and non parametric regression

For linear and parametric regression there are multiple tests where variables and residuals are used by means of performing a linear regression function to test serial correlation of regression errors ...
1
vote
1answer
68 views

Biasing SkLearn Algorithms to Positive Outcomes

I am trying to run multinomial naive bayes on a series of examples in python using sci kit learn. I am consitently getting all examples classified as negative. (The ratio of positives to negatives in ...
4
votes
1answer
253 views

Which algorithm should I use?

I was reading many machine learning questions like this one but I am not sure how to apply them to my scenario. I come from a biology/medicine background, and my math knowledge is limited (last thing ...
14
votes
2answers
8k views

What are the advantages of ReLU over sigmoid function in deep neural network?

The state of the art of non-linearity is to use ReLU instead of sigmoid function in deep neural network, what are the advantages? I know that training a network when ReLU is used would be faster, and ...
3
votes
1answer
287 views

Optimal construction of day feature in neural networks

Working on regression problem I started to think about representation of "day of a week" feature. I wonder which approach would perform better: one feature; value 1/7 for Monday; 2/7 for Tuesday... ...
-1
votes
1answer
340 views

Distribution Assumptions in Ridge & Lasso Regression Models?

What are the assumptions for the distribution of the features for regression models like Lasso regression or Ridge regression? Why is it better to have features with Gaussian distributions?
4
votes
1answer
44 views

Is there such thing as regression involving a pairwise response variable? (X,Y)~Z0+Z1*B1

I'm trying to model a pairwise outcome of basketball game scores. Ie. (94,87),(102,98),(76,54),... My input variable is a single performance metric for each team. Ie. (12,9),(14,17),... Is there a ...
0
votes
0answers
168 views

credit scoring - fraud scoring

I have been asked to build a credit scoring model and we are relying on several Machine Learning API, in order to build our feature vectors. One of these API is MinFraud. However, as they provide us ...
1
vote
1answer
398 views

Convolutional neural network - Using absolute of tanh on convolution output

I've watched an online lecture regarding CNN (https://www.youtube.com/watch?v=wORlSgx0hZY) that confused me a bit. At roughly 8:35 in the lecture it was stated that it is important to use the absolute ...
0
votes
1answer
114 views

If random forests gives me a bad cross-validation score, should I trust it for feature selection?

I get an R^2 value of about 0.22 when I 10-fold cross-validate with my entire dataset. My main use for random forests is to analyze feature interactions. But should I trust the feature importances ...
0
votes
1answer
269 views

Complement naive bayes

I would like some help in understanding how does Complement Naive Bayes work. I have googled the paper Complement Naive Bayes I understand that naive bayes works by computing the probability of a ...
0
votes
0answers
31 views

Machine Learning: Potential Reasons of Precision Change after New Features are Added

My baseline model uses 10 features $[f_1, f_2, \dotsb, f_{10}]$. Now I have two new features $f_{11}$ and $f_{12}$. New models that use either $[f_1, f_2, \dotsb, f_{10}, f_{11}]$ or $[f_1, f_2, \...
1
vote
0answers
24 views

Can summation of gradient descent runs on subsets approximate gradient descent run on set?

I have a training set M, which I then split into two subsets, say, M1 & M2. If I run gradient descent on each subset to come up with a linear regression which approximates the data in M1 & M2,...
0
votes
2answers
30 views

For the given type of dataset, what would generally be the set of classifiers that should be tried to get the highest TPR for FPR = 0.01

I'm primarily looking to attain the maximum True Positive Rate for a small False positive Rate of say 0.01. The following is an instance: 1, 37.33, 228.39, 0, 77.060599, 0.073384, 0.052536, 1.389826, ...
1
vote
1answer
100 views

Step training Baum-welch HMM

Referred to Baum–Welch algorithm, http://cs.au.dk/~cstorm/courses/MLiB_f14/slides/hidden-markov-models-4.pdf Is this formula correct? I spend a couple days to figure out which part is wrong. I'm ...
3
votes
1answer
490 views

Spatial coordinates (latitude and longitude) are non significant

I want to use latitude and longitude as a feature for models like SVM or Logistic Regression (both for classification). What is the most common approach to use latitude and longitude values as ...
1
vote
0answers
136 views

Cross-Validation: how to choose k for small datasets (n=900)?

In a binary classification task, I have a small training set (n=900, 9 features). The two groups are not symmetric (1 = 560, 0 = 340). I also have a test set (n=400) where I don't know the class ...
1
vote
0answers
95 views

How to choose the right step size for alpha in the elastic net?

I'm using "glmnet" package in R to learn different elastic net regressions. As you know, elastic net should perform at least as good as LASSO regression. But it's not the case for me and LASSO perform ...
1
vote
1answer
74 views

Result of K-Means Algorithm Not Desired

I am learning about K-means algorithm, and I have generated a dataset with 150000 data points, with 10000 points per cluster. (Scatter plot at the bottom) When I run K-means on the dataset, I first ...
1
vote
2answers
107 views

'Punishment Function' in Number of Knots in Splines?

I was considering using natural cubic splines for my prediction problem when I had a thought: In Ridge Regression, you set out to minimize the equation; \begin{equation} F(X)=\lambda\sum_i ( b^2)+ \...
1
vote
2answers
524 views

How to bootstrap panel data?

I'm fitting some machine learning algorithms (e.g. SVM) on my panel data. It's taking too long for my entire dataset, so I'm considering generating smaller samples from bootstrapping then fit the SVM ...
0
votes
0answers
46 views

R: “weights” option will help calibrating class inequality?

I have a database with a binary response variable and 100 predictors (correlated and uncorrelated). I want to try the machine learning techniques in R I've been reading about in the last 3 weeks (...
1
vote
0answers
272 views

Deriving the maximum likelihood for a generative classification model for K classes

In Christopher Bishop's book "Pattern Recognition and Machine learning", there is the following question: Consider a generative classification model for $K$ classes defined by the prior class ...
0
votes
1answer
273 views

What is Nadaraya-Watson Kernel Regression Estimator for Multivariate Response?

Given a regression setting with covariates $X_{n \times m}$ and response $Y_{n \times p}$ where $p>1$, i.e the responses are vector-valued or multivariate, is there a Nadaraya-Watson estimator for ...
5
votes
1answer
290 views

Conditional independence iff joint factorizes

I have proven that: $X⊥Y|Z\ {\rm iff}\ p(x,y|z)=p(x|z)p(y|z)$ for all $x,y,z$ such that $p(z)>0$. The next question is to prove an alternative definition: $X⊥Y|Z$ iff there exist functions $g$ ...
1
vote
2answers
563 views

How to find optimal penaltyparameter C for SVM (regression)

I am training an svm regressor using python sklearn.svm.SVR From the example given on the sklearn website, the above line of code defines my svm. ...
1
vote
0answers
45 views

Machine Learning Predictors Evaluation Using R

I've bee using R for predicitve analytics and here is issue: I'm trying to predict the species (categorical variables E1, E2, E3 and E4) of an animal using as predictors a set of nominal (NO1, NO2, ...
0
votes
1answer
41 views

SVM parameters clarification

James et al. in An introduction to the statistical learning (p. 351) claim that the solution to the support vector classifier problem involves only the inner products of the observations. They ...
0
votes
1answer
84 views

After Clustering, how can I evaluate which features had the biggest impact?

I've just performed unsupervised clustering (using DBSCAN) on a dataset for which I have no expert knowledge on. I'm interested in working out which features had the greatest impact on my clustering. ...
1
vote
0answers
26 views

How do I find corresponding clusters in independent samples?

Lets suppose you believe that observations in your data come from K natural but not directly observable categories and you wish to identify these categories with minimal prior assumptions, so you find ...
1
vote
1answer
875 views

Machine learning algorithm to predict next user's destination

I'm searching for a way to formulate my problem as a machine learning problem. Suppose I have a history of user's locations, and I want to predict his next location, similar to how Google Now does it ...
0
votes
1answer
63 views

Why do two identical feature vectors (distance score 0) get different labels in DBSCAN?

I have two identical feature vectors. They have a distance score of 0. I perform DBSCAN Clustering (using sci-kit) and they get different labels. Is this expected behaviour?
2
votes
2answers
112 views

Backward feature selection with CV model selection

I am thinking about doing the following to a data set with $N$ samples and $m$ features 1) Train using semi-supervised learning and cross validate on labeled data using LOO-CV to select the best ...
2
votes
1answer
88 views

Is There A Machine Learning Algorithm For Textual Data With Thousands Of Classifiers?

I've been asked to migrate this from StackOverflow to CrossValidated. I have a problem that I think Machine Learning can solve but am having a very hard time determining which ML Algorithm to use and ...
0
votes
0answers
184 views

Handling Sparse Data Frames - algorithm selection

I am new to machine learning/statistical modelling. I am trying to run a classification on a highly sparse dataset with 100 features, most of which are categorical (TRUE/FALSE) with the remaining ...
-1
votes
2answers
176 views

k-mean clustering of week-times

I have data of meeting times. The data has weekday and hour of the day. I want to cluster the meeting times (I have reason to believe there are two different kinds of meetings that tend to occur at ...
1
vote
0answers
59 views

Data At Varying Granularity

I'm sorry for asking such a simple question, but for some reason it is throwing me off. By "granularity" I mean level of the data. For example, say in the classic example of spam classification you ...
1
vote
0answers
87 views

why do offline learning algorithms perform better than their online learning counterparts?

(I'm assuming infinite data, finite time for this comparison) I was wondering why it is exactly that online learning algorithms usually perform more poorly than their offline counter-parts.. Does ...
0
votes
1answer
41 views

Analysis of Feature Importances when features are dependent on one another

I can use random forests to determine which features are important when doing a prediction problem; for example. < height, weight, IQ measure> -> Is considered obese? Applying random forests ...
1
vote
2answers
1k views

PCA on train and test datasets: should I run one PCA on train+test or two separate on train and on test? [duplicate]

I'm doing an image classification task and the number of features of each example image is pretty huge (3,072: # pixels in each image). I'm thinking of using PCA to reduce the # features of each image ...
1
vote
0answers
46 views

How to estimate the correlated individual components from a sum, for a random process?

Assume that there are $N$ realisations of five individual, random variables$X_1$, $X_2$, $X_3$, $X_4$ and $X_5$, which in general could be correlated. We define another random variable $S=X_1+X_2+X_3+...
5
votes
1answer
71 views

Understanding sample complexity in the context of uniform convergence

I was reading Andrew Ng's notes and on page 6 he mentions (uniform convergence): $$Pr[\forall h \in \mathcal{H}_{finite}|\epsilon(h_i)-\hat{\epsilon}(h_j)| \leq \gamma] \geq 1-2ke^{-2\gamma^2m}$$ ...
0
votes
0answers
132 views

In Hidden Markov Model (HMM), is the transition matrix known, inferred, or assumed?

I'm reading Kevin Murphy's Probabilistic Machine Learning, which explains the forward algorithm to do filtering in HMM as follows (pp 610): The very first line says that the transition matrices $\...
2
votes
1answer
105 views

Number of variables for decision trees

I have a data with just 5 independent variables and a response. I am dealing with a classification problem. Will decision trees perform well or the number of variables have to be higher to get ...