Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

5,902 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
0answers
109 views

Derive the gradients of a basic neural network

Given a neural network as following \begin{align*} &J = CE(y,\hat{y})=-\sum_i y_i log(\hat{y}_i)\\ &\hat{y} = softmax(z_2)\\ &z_2 = hW_2+b_2\\ &h = sigmoid(z_1)\\ &...
4
votes
0answers
188 views

How do you evaluate machine learning model already deployed in production?

so to be more clear lets consider the problem of loan default prediction. Let's say I have trained and tested off-line multiple classifiers and ensembled them. Then I gave this model to production. ...
4
votes
0answers
534 views

What is the difference between an association rule and Pearson's correlation?

In its most common form, association rule learning involves a collection of transactions. For each transaction, there is a set of possible items present. If an item is present in a transaction, then $...
4
votes
0answers
166 views

Sparse Representation, Sparse Learning, Sparse Coding, Group Sparse Coding and Group Sparse Learning?

I'm really confused with these terms for the relations and difference between them: Sparse Representation Sparse Learning Sparse Coding Group Sparse Coding Group Sparse Learning Sparse Dictionary ...
4
votes
0answers
563 views

How does a maze game / directed paths map to a Neural Network?

I am trying to understand how a maze game's possible paths and possible moves are mapped out in a Neural Network. Let's take this example here: http://cdn.intechopen.com/pdfs-wm/10916.pdf Agent can ...
4
votes
1answer
1k views

In Machine Learning, how does getting more training examples fix high variance $(Var(\hat f(x_{0})))$?

I don't believe that (Why does increasing the sample size lower the variance?) appropriately handles my question! The linked questions explains why any addition of random variables (all iid) produces ...
4
votes
0answers
444 views

feature scaling in online learning algorithms

Let's take Logistic Regression as an example, typically we should do feature scaling before applying l1 or l2 regularizations. ...
4
votes
1answer
134 views

Bayes net probability question

I've made this Bayes net based on a problem and I'm trying to find the probability of W but I'm stuck. I know I probably have to use Bayes theorem backwards through to find $P(W)$, but I'm not sure ...
4
votes
0answers
101 views

Can I use HMM to predict the spread of Ebola?

1) Can Hidden Markov Model be used across both a large number of categories (districts) and cases (weeks)? 2) Is HMM appropriate for trying to model such a problem? 3) Would I need to develop a ...
4
votes
0answers
798 views

A mixture of conjugate priors is conjugate

I want to prove that a mixture of conjugate priors is itself conjugate. It does not look difficult, but I'm still a bit unsure when manipulating probabilities, especially in a Bayesian context. Is ...
4
votes
0answers
552 views

Normalizing data worsens the performance of CNN?

I've been using CNN for facial recognition tasks, first I train a CNN for classification, and I use the trained CNN to extract features from images and do verification (tell whether two pictures are ...
4
votes
1answer
663 views

Likelihood of LDA compared to logistic regression

I've come across an interesting exercise. We are given four classification models for binary response and a $d$-dimensional independent variable: A Linear Discriminant Analysis model where the ...
4
votes
0answers
90 views

Training instances importance in Random Forest?

Is it possible to determine the importance of the training examples in Random Forests, analogously to what's done with predictors? Basically the idea would be to find important samples in the data, ...
4
votes
0answers
10k views

Splitting data for train/test for time series

A week ago or so I was at a conference. Long story short, I ran into a friend who is quite good at machine learning so I asked them a question about why I might be getting what I think is poor fit on ...
4
votes
0answers
737 views

Gradient descent vs Contrastive Divergence

What are the differences (if any) between gradient descent and Contrastive Divergence? I understand how gradient descent is used to train neural networks via back-propogation, but I've just started ...
4
votes
0answers
211 views

Calculating conditional probability in Bernoulli mixture model

I'm following the book Pattern recognition and machine learning by Bishop on Bernoulli mixture model, and trying to code it. But I don't understand how to calculate the conditional probability (page ...
4
votes
0answers
523 views

Counter intuitive behavior from scikit-learn's SGDClassifier

I am working with SGDClassifier from Python library scikit-learn, a function which implements linear classification with a ...
4
votes
0answers
354 views

Bag of Features / Visual Words + Locality Sensitive Hashing

PREMISE: I'm really new to Computer Vision/Image Processing and Machine Learning (luckily, I'm more expert on Information retrieval), so please be kind with this filthy peasant! :D MY APPLICATION: ...
4
votes
0answers
473 views

How to obtain a confidence interval or a measure of prediction dispersion when using xgboost for classification?

How to obtain a confidence interval or a measure of prediction dispersion when using xgboost for classification? So for example, if xgboost predicts a probability of an event is 0.9, how can the ...
4
votes
0answers
652 views

Can (loopy) belief propagation be used to learn from a data set?

I'm trying to expand my experience with restricted Boltzmann machines to a more general class of graphical models and currently learning about belief propagation using message passing algorithms. One ...
4
votes
0answers
192 views

Machine Learning and Flow Maximization

Has anyone ever seen machine learning (ML) used to assist a Max Flow algorithm? I have a very large directed graph that has some fractal characteristics, meaning that this large graph can be roughly ...
4
votes
0answers
147 views

How to mitigate the hierarchical error propagation in tree-structured classification

Suppose we have a multi-class classification problem, where the number of classes $K \geq 3$ We use a tree structure of multiple SVMs to divide and conquer the problem, with one example in the figure ...
4
votes
0answers
585 views

How to tune the weak learner in boosted algorithms

It is commonly said that boosted algorithms (adaboost, gradient boosted trees) are composed of many "weak" learners. Let's stick to decision trees as the base learners. Some empirical studies ...
4
votes
3answers
2k views

Combining one class classifiers to do multi-class classification

I am working on a 3-class classification problem. The classifier I'm using is Bayesian Networks which provides me with a classification accuracy of around 60%. When I do a two-class classification, I ...
4
votes
0answers
718 views

Understanding calibrating probabilities using R

I am trying to understand R's calibration(package:caret) function. My main interest is binary classification. Calibration function is used for plotting true ...
4
votes
0answers
807 views

Ill-conditioned covariance matrices in EM

I am currently working with the Expectation-Maximization algorithm. I have some pre-clustered sets of 3D points and am trying to run the algorithm. However I've seen that most of my covariance ...
4
votes
0answers
1k views

Nesterov vs. momentum gradient descent

I implemented these two methods in a deep learning project where I am using theano. I understand the mathematical difference between these two methods, and my conceptual understanding is that ...
4
votes
0answers
2k views

How to know when to use Kernel SVM and not Linear SVM?

If I have more than 3 features in a dataset, then I can't visualize them to see if my classes are scattered in a non linear fashion. So how do I know when is the right way to use linear model with non-...
4
votes
0answers
797 views

How to estimate a probability distribution

Suppose I want to estimate a probability distribution, is it common practice to simply fit a function to a frequency histogram? So in my work, I am training a classifier, the performance of which is ...
4
votes
1answer
495 views

A statistical test to measure the importance of features?

I'm currently trying to assess importance of the features for my classifier. The situation is the following: first I train my classifier with all of the features I have and tested on a test set . Then ...
4
votes
0answers
1k views

Random Forest online/incremental learning in R

Is there a Random Forest implementation available in R, that supports online learning? My alternative approach was to use the popular randomForest package and combine Random Forests (the existing one ...
4
votes
0answers
249 views

full conditional posteriors for bayesian lasso

I am reading the original Bayesian Lasso paper, and its follow up; They look straightforward to implement, mainly because of the conditional posterior probability for the gibbs sampler; however, I ...
4
votes
0answers
1k views

How to draw plot of the values of decision function of multi class svm versus another arbitrary values?

I am trying to draw a plot of the decision function ($f(x)=sign(wx+b)$ which can be obtain by fit$decision.values in R using the svm function of e1071 package) versus another arbitrary values. From ...
4
votes
0answers
516 views

Decomposing the non-deterministic transition functions in non-Markov decision processes into several deterministic transition functions

Problems in reinforcement learning are commonly modeled as Markov decision processes (MDPs). One essential part of MDPs is the transition function $T: S \times A \times S \rightarrow [0, 1] \in \...
4
votes
0answers
248 views

Modelling of probabilistic vs deterministic systems

The learning problem in Statistical Learning Theory is defined as: $$ R(f) = \int_{X,Y} L(y, f(x))P(x,y)\mathrm{d}x\mathrm{d}y $$ where $R(f)$ is the expected risk $L$ is the loss function $P(x, y)...
4
votes
0answers
743 views

Why does this multi-response Guassian LASSO not give a sparse solution?

I tried the glmnet package to learn multi-response Gaussian family. I have looked at the coefficients of the final model. The result is odd. All the features have ...
4
votes
0answers
1k views

When to use the Kappa statistic evaluation metric?

Can someone tell me when is it appropriate to use the Kappa statistic? Also why to use it when one can use Area Under the ROC curve? Or even the Area under the precision-recall curve? So what are the ...
4
votes
0answers
2k views

Deriving the maximum likelihood for a generative classification model for K classes

In Christopher Bishop's book "Pattern Recognition and Machine learning", there is the following question: Consider a generative classification model for $K$ classes defined by the prior class ...
4
votes
0answers
652 views

How to understand kernel functions and how to choose a suitable kernel?

I am trying to describe my understand of kernels in the Support Vector Machine(SVM) and why some of them are more popular, but I am not sure if I misunderstand these concepts: 1) There are a large ...
4
votes
0answers
252 views

Duda, Hart, Stork No Free Lunch Discussion

Please see this question regarding Duda, Hart, and Stork's No Free Lunch Theoremm Discussion Hi all, I was having trouble understanding the description of the NFL theorem in Duda, Hart, and Stork. My ...
4
votes
0answers
126 views

Vapniks proof of the basic lemma

In his book Statistical Learning Theory (1998), Vladimir Vapnik proves an inequality needed to prove a bound on the risk for indicator loss functions. Theorem 4.1 on page 133 he derives the following ...
4
votes
0answers
928 views

Learning to Rank: query-dependent vs. query-independent features

I've been doing some reading about learning to rank - specifically lambdaMART - and one thing I am confused about is the role of features. When training a model, should one only use query-dependent ...
4
votes
0answers
172 views

Record linkage when sources have different fields

I have read a little about record linkage, but it seems to me that a requirement is that all fields in both sources can be compared. For example, with sources A and B, an assumption is that we can ...
4
votes
0answers
266 views

Time series prediction where each datapoint has a sequence

I am Computer Science major, and new to stats, so please bear with me and point me to the right direction if what I'm asking is pretty obvious. I have a dataset, where each data point consists of <...
4
votes
1answer
496 views

How to estimate False Discovery Rate from p-value distribution?

I have learned many models and I calculated p-values for the cross-validation errors. I want to select significant models based on the false discovery rate (FDR). How can I estimate the FDR from p-...
4
votes
0answers
539 views

Reducing size of dataset to a fixed size - retaining maximum information in all dimensions

I was wondering about about the following problem: I have a set of $N=10^5$ observations with dimensionality $D=2$, and I would like to reduce it to a set of size with $M=10^3$, or some other $(M \ll ...
4
votes
0answers
368 views

Linear Discriminant Analysis: Using subject as classification

I have a problem where I need to identify from which subject a particular set of data points came. More specifically, my problem is that I need to demonstrate that my subjects (N=9) can be ...
4
votes
0answers
1k views

Prediction using SVD and Fisher's linear discriminant

Where can I get an explanation of the procedure used when making a prediction using SVD? Let me elaborate a bit more. Suppose you have data in a matrix $A$ containing two classes. In particular, you ...
4
votes
0answers
3k views

How to train SVM correctly on a 1D dataset

I am trying to use svmtrain (Statistic Toolbox) to train a linear (2 class) SVM on a 1D feature vectors. The features are not fully separable and the classes intersect. The naive approach would be ...