Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms (1)

0
votes
1answer
10 views

What is Nadaraya-Watson Kernel Regression Estimator for Multivariate Response?

Given a regression setting with covariates $X_{n \times m}$ and response $Y_{n \times p}$ where $p>1$, i.e the responses are vector-valued or multivariate, is there a Nadaraya-Watson estimator for ...
2
votes
0answers
20 views

Conditional independence iff joint factorizes

I have proven that: $X⊥Y|Z\ {\rm iff}\ p(x,y|z)=p(x|z)p(y|z)$ for all $x,y,z$ such that $p(z)>0$. The next question is to prove an alternative definition: $X⊥Y|Z$ iff there exist functions $g$ ...
0
votes
1answer
14 views

How to find optimal penaltyparameter C for SVM (regression)

I am training an svm regressor using python sklearn.svm.SVR From the example given on the sklearn website, the above line of code defines my svm. ...
0
votes
0answers
12 views

Machine Learning Predictors Evaluation Using R

I've bee using R for predicitve analytics and here is issue: I'm trying to predict the species (categorical variables E1, E2, E3 and E4) of an animal using as predictors a set of nominal (NO1, NO2, ...
0
votes
0answers
6 views

Python “feature_importances” for most important factors

I'm a little unsure as whether this belongs in stackoverflow or cross validated. I have found a few posts on this topic , but I have not found the following question. Is it accurate to run the feature ...
0
votes
1answer
20 views

SVM parameters clarification

James et al. in An introduction to the statistical learning (p. 351) claim that the solution to the support vector classifier problem involves only the inner products of the observations. They ...
0
votes
0answers
19 views

After Clustering, how can I evaluate which features had the biggest impact?

I've just performed unsupervised clustering (using DBScan) on a dataset for which I have no expert knowledge on. I'm interested in working out which features had the greatest impact on my clustering. ...
1
vote
0answers
15 views

How do I find corresponding clusters in independent samples?

Lets suppose you believe that observations in your data come from K natural but not directly observable categories and you wish to identify these categories with minimal prior assumptions, so you find ...
0
votes
1answer
31 views

Machine learning algorithm to predict next user's destination

I'm searching for a way to formulate my problem as a machine learning problem. Suppose I have a history of user's locations, and I want to predict his next location, similar to how Google Now does it ...
0
votes
1answer
10 views

Why do two identical feature vectors (distance score 0) get different labels in DBSCAN?

I have two identical feature vectors. They have a distance score of 0. I perform DBSCAN Clustering (using sci-kit) and they get different labels. Is this expected behaviour?
0
votes
2answers
46 views

Backward feature selection with CV model selection

I am thinking about doing the following to a data set with $N$ samples and $m$ features 1) Train using semi-supervised learning and cross validate on labeled data using LOO-CV to select the best ...
2
votes
0answers
33 views

Is There A Machine Learning Algorithm For Textual Data With Thousands Of Classifiers?

I've been asked to migrate this from StackOverflow to CrossValidated. I have a problem that I think Machine Learning can solve but am having a very hard time determining which ML Algorithm to use and ...
0
votes
0answers
25 views

Chi square and zscore - chose which one?

I'm newbie in machine learning. Recently I tried to learn something on this and got following concern: I have products classed by categories. Also I have users with gender and device model ...
0
votes
0answers
30 views

Advice for feature selection or feature extraction with semi-supervised learning

I am trying to solve a semi-supervised learning problem using LaplacianSVM. However, before applying LapSVM I would like either to perform feature selection or feature extraction. Furthermore, after ...
0
votes
0answers
24 views

Handling Sparse Data Frames - algorithm selection

I am new to machine learning/statistical modelling. I am trying to run a classification on a highly sparse dataset with 100 features, most of which are categorical (TRUE/FALSE) with the remaining ...
0
votes
2answers
41 views

k-mean clustering of week-times

I have data of meeting times. The data has weekday and hour of the day. I want to cluster the meeting times (I have reason to believe there are two different kinds of meetings that tend to occur at ...
1
vote
0answers
12 views

Data At Varying Granularity

I'm sorry for asking such a simple question, but for some reason it is throwing me off. By "granularity" I mean level of the data. For example, say in the classic example of spam classification you ...
1
vote
0answers
30 views

why do offline learning algorithms perform better than their online learning counterparts?

(I'm assuming infinite data, finite time for this comparison) I was wondering why it is exactly that online learning algorithms usually perform more poorly than their offline counter-parts.. Does ...
0
votes
0answers
16 views

Analysis of Feature Importances when features are dependent on one another

I can use random forests to determine which features are important when doing a prediction problem; for example. < height, weight, IQ measure> -> Is considered obese? Applying random forests ...
1
vote
2answers
43 views

Multi-class Classification using SVM with PCA

I'm doing an image classification task and the number of features of each example image is pretty huge (3,072: # pixels in each image). I'm thinking of using PCA to reduce the # features of each image ...
1
vote
0answers
30 views

How to estimate the correlated individual components from a sum, for a random process?

Assume that there are $N$ realisations of five individual, random variables$X_1$, $X_2$, $X_3$, $X_4$ and $X_5$, which in general could be correlated. We define another random variable ...
5
votes
1answer
35 views

Understanding sample complexity in the context of uniform convergence

I was reading Andrew Ng's notes and on page 6 he mentions (uniform convergence): $$Pr[\forall h \in \mathcal{H}_{finite}|\epsilon(h_i)-\hat{\epsilon}(h_j)| \leq \gamma] \geq 1-2ke^{-2\gamma^2m}$$ ...
0
votes
0answers
18 views

In Hidden Markov Model (HMM), is the transition matrix known, inferred, or assumed?

I'm reading Kevin Murphy's Probabilistic Machine Learning, which explains the forward algorithm to do filtering in HMM as follows (pp 610): The very first line says that the transition matrices ...
2
votes
1answer
23 views

Number of variables for decision trees

I have a data with just 5 independent variables and a response. I am dealing with a classification problem. Will decision trees perform well or the number of variables have to be higher to get ...
1
vote
1answer
37 views

Combining pca and classification algorithms

For some classification algorithms, assuming independence of data helps reduce the number of parameters to estimate. Why then not just to apply a method like pca or ica to the original features to get ...
0
votes
1answer
26 views

Machine learning : learn feature value range for a classification

Which domain the problem belongs to? Given a set of products some are classified as cheap and some not. The task is to determine the price range (probablistic) for cheap products ? Supervised ...
0
votes
1answer
35 views

Machine Learning for Text Classification

I am new to Machine Learning.I am working on a project where the machine learning concept need to be applied. Problem Statement: I have large number(say 3000)key words.These need to be classified ...
0
votes
0answers
31 views

Unsupervised learning algorithems to detect anomaly in waves

I have a sample of graphs (more then 10000...). that look like in the image below: I am searching an Unsupervised learning algorithems thet can help me to detect Anomaly observations. Here what i ...
0
votes
0answers
22 views

How can one go about recognizing a kind of motion using 3D depth data?

I'm using a Kinect device, and I'm currently extracting Joints, and Depth data unto probably a buffer data of 15 frames. This is done at 30 frames per second. The whole point of it is to try and ...
0
votes
0answers
26 views

Making a neural network model more sensitive to one of its several inputs

I am currently using neural network methods in R to model energy consumption (response) based on temperatures, previous consumption values and weekend dummy variables (inputs). Unfortunately, the ...
0
votes
0answers
25 views

SVM Dual Formulation :: KKT Constraint

In Andew Ng's SVM course notes, the final hard margin optimization problem is given as the following: I am unclear how to see from this where the 5th constraint is satisfied. The definition of ...
0
votes
0answers
11 views

Support vector regression in weka

I am using SVR for statistical down-scaling of precipitation. I have taken the first 3 factor scores in principal component analysis of variables as predictors and precipitation as predictand. As ...
0
votes
1answer
42 views

In natural language processing (NLP), how do you make an efficient dimension reduction?

In NLP, it's always the case that the dimension of the features are very huge. For example, for one project at hand, the dimension of features is almost 20 thousands (p = 20,000), and each feature is ...
0
votes
0answers
26 views

Gradients of marginal likelihood of Gaussian Process with squared exponential covariance, for learning hyper-parameters

The derivation of gradient of the marginal likelihood is given in this pdf, equation 5.9. But the gradient for the most commonly used covariance function, squared exponential covariance, is not ...
2
votes
1answer
72 views

How to estimate the individual components from a sum for a random process?

We have $N$ realisations of five individual, IID random variables $X_1$, $X_2$, $X_3$, $X_4$ and $X_5$. We define another random variable $S = X_1+X_2+X_3+X_4+X_5$. Now, for a given $S$ generated from ...
2
votes
0answers
26 views

Connections between d' (d-prime) and AUC (Area Under the ROC Curve); underlying assumptions

In machine learning we may use the area under the ROC curve (often abbreviated AUC, or AUROC) to summarise how well a system can discriminate between two categories. In signal detection theory often ...
3
votes
2answers
45 views

Mining patterns in continuous sequence

I have data in form of $N$ sequences $s_j=(t_i, e_i)_{i\in\{1,\ldots,n_j\}}$ with $n_j$ data-points each, where $t_i$ is a time-stamp and $e_i$ is a categorial event, say $e_i\in\{A,B,C,D\}$. The $N$ ...
0
votes
0answers
9 views

Is it OK to resample data in one vs all multiclass classification?

I have a multiclass classification problem and decide to use one vs all logistic regression. Since some classes are very rare (pos vs neg is like 1:100), I plan to use some balancing strategy during ...
8
votes
1answer
108 views

Why is the definition of a consistent estimator the way it is? What about alternative definitions of consistency?

Quote from wikipedia: In statistics, a consistent estimator or asymptotically consistent estimator is an estimator—a rule for computing estimates of a parameter $θ^*$—having the property that ...
0
votes
0answers
29 views

Adding weights to data points in logistic regression

I am trying to run a logistic regression on a data sample where the unique identifier is "project". I also have the date on which each project was created. Some projects are more recent than others ...
0
votes
0answers
10 views

Normalisation formula applicable to one or more data items

I've created a recurrent neural network, to which normalised values are passed as inputs. The normalization formula is: $$\tilde{x_{i}} = \frac{1}{1+exp(-\frac{x_i-\bar{x}}{\sigma})},$$ where ...
0
votes
0answers
13 views

Using PyBrain after training a network

I'm using PyBrain to create a neural network. I'm still pretty new to neural networks and their concepts. I've so far only run train() over the network, as ...
1
vote
1answer
15 views

How to get random classification to assess the performance of classifier with McNemar test?

I'm trying to replicate a study where the author used the McNemar test to assess the performance of classification compared to random classification. I have the original classifier and I'm using R to ...
1
vote
1answer
36 views
+50

How to categorize classifiers and matrix factorization methods?

I have a classification problem which is solved by a variety of methods. Among the methods are unsupervised methods, traditional classifiers and a supervised matrix factorization methods. The problem ...
1
vote
1answer
23 views

How do I incorporate the biases in my feed-forward neural network

I'm trying to implement a FFNN. I'm doing this as an excercise to understand how biases play a role in the classification. I trained a NN using a package in R with the inputs being 1..100 and the ...
1
vote
2answers
40 views

Why Adaboost with Decision Trees?

I've been reading a bit on boosting algorithms for classification tasks and Adaboost in particular. I understand that the purpose of Adaboost is to take several "weak learners" and, through a set of ...
1
vote
1answer
66 views

Gaussian is conjugate of Gaussian?

Someone told me that, Gaussian distribution is conjugate to distribution because a Gaussian times a Gaussian would still be Gaussian distribution ? Why is that ? Say the following situation: $X\sim ...
0
votes
0answers
13 views

Predicting the near-future values using an unevenly sampled time-series data

Summary Need help with predicting the near-future values using an unevenly sampled time-series data. Data is collected as events, and is converted to time series. I have tried out a few approached ...
1
vote
0answers
15 views

Why all coeficents of features of model are zero while I have high deviance using glmnet?

I'm using gmlnet to learn lasso regression model. model<-cv.glmnet(x, y, alpha=1, nfolds=10,parallel= TRUE) when I learn model and look at the model it's like this : ...
0
votes
0answers
22 views

Machine learning and Partial differential equations

Are there any algorithms which were developed using partial differential equations for tackling some of the machine learning problems? Most works I see online are in the field of computer vision and a ...