Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

4,453 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
1answer
612 views

the role of basis functions in reinforcement learning

In the very simple examples of reinforcement learning (gridworld, mountain car), we use real numbers or some elementary functions as reward functions. When state spaces become larger and larger, and ...
4
votes
0answers
78 views

Sparse vs compact representations

In sparse representations, we like to find representation of the input where most elements are nearly zero. On the other hand, in some applications we prefer dense representations such as word ...
4
votes
1answer
59 views

In variational inference on von Mises clusters, how to find a bound for the log-Bessel function?

This paper on von Mises clustering uses an upper bound on the modified log-Bessel function that I struggle to replicate. Taking results from this paper, the authors state: $$u\frac{I'_\nu(u)}{I_\nu(u)...
4
votes
0answers
398 views

Train Neural Network For Handwritten Chinese Characters

The article here: http://novanoid.github.io/2014/09/26/training-a-neural-network-to-recognize-handwritten-digits/ discusses and implements a way to recognize handwritten digits. For images with a ...
4
votes
0answers
227 views

Ensemble learning with time-varying covariates and effects

We are interested in replicating several duration studies in the literature using ensemble learning methods. After some experimentation, we opted for random survival forests (Ishwaran et al. 2008) for ...
4
votes
1answer
1k views

Calculate the decision boundary for Quadratic Discriminant Analysis (QDA)

I am trying to find a solution to the decision boundary in QDA. The question was already asked and answered for LDA, and the solution provided by amoeba to compute this using the "standard Gaussian ...
4
votes
0answers
181 views

How do you evaluate machine learning model already deployed in production?

so to be more clear lets consider the problem of loan default prediction. Let's say I have trained and tested off-line multiple classifiers and ensembled them. Then I gave this model to production. ...
4
votes
0answers
305 views

Unsupervised Anomaly Detection Threshold Selection

If we have a data set that contains only positive examples I am wondering how we can effectively choose a threshold for an anomaly detection technique. Are there anomaly detection techniques that can ...
4
votes
0answers
148 views

Sparse Representation, Sparse Learning, Sparse Coding, Group Sparse Coding and Group Sparse Learning?

I'm really confused with these terms for the relations and difference between them: Sparse Representation Sparse Learning Sparse Coding Group Sparse Coding Group Sparse Learning Sparse Dictionary ...
4
votes
0answers
491 views

How does a maze game / directed paths map to a Neural Network?

I am trying to understand how a maze game's possible paths and possible moves are mapped out in a Neural Network. Let's take this example here: http://cdn.intechopen.com/pdfs-wm/10916.pdf Agent can ...
4
votes
1answer
600 views

In Machine Learning, how does getting more training examples fix high variance $(Var(\hat f(x_{0})))$?

I don't believe that (Why does increasing the sample size lower the variance?) appropriately handles my question! The linked questions explains why any addition of random variables (all iid) produces ...
4
votes
1answer
96 views

Bayes net probability question

I've made this Bayes net based on a problem and I'm trying to find the probability of W but I'm stuck. I know I probably have to use Bayes theorem backwards through to find $P(W)$, but I'm not sure ...
4
votes
0answers
197 views

Sample Space of Machine Learning Classification “Experiment”

If you're trying to classify some input, $\mathbf{x} \in \mathbb{R}^{n}$, to one of $d$ classes using a model with parameters, $\theta$, how are you supposed to think about the experiment of learning ...
4
votes
0answers
96 views

Can I use HMM to predict the spread of Ebola?

1) Can Hidden Markov Model be used across both a large number of categories (districts) and cases (weeks)? 2) Is HMM appropriate for trying to model such a problem? 3) Would I need to develop a ...
4
votes
1answer
529 views

Likelihood of LDA compared to logistic regression

I've come across an interesting exercise. We are given four classification models for binary response and a $d$-dimensional independent variable: A Linear Discriminant Analysis model where the ...
4
votes
0answers
85 views

Training instances importance in Random Forest?

Is it possible to determine the importance of the training examples in Random Forests, analogously to what's done with predictors? Basically the idea would be to find important samples in the data, ...
4
votes
0answers
7k views

Splitting data for train/test for time series

A week ago or so I was at a conference. Long story short, I ran into a friend who is quite good at machine learning so I asked them a question about why I might be getting what I think is poor fit on ...
4
votes
1answer
256 views

How to make sense of the EM algorithm expressed in terms of Kullback-Leibler divergence?

In the textbook All of Statistics by Wasserman, the algorithm is expressed as: Pick a starting value $\theta^0$. (E-Step). Calculate: $$ J(\theta|\theta^j) = E_{\theta^j} \left(\log \dfrac{f(Y^n, Z^...
4
votes
0answers
139 views

How to mitigate the hierarchical error propagation in tree-structured classification

Suppose we have a multi-class classification problem, where the number of classes $K \geq 3$ We use a tree structure of multiple SVMs to divide and conquer the problem, with one example in the figure ...
4
votes
2answers
1k views

How do I find multiple change points in an online dataset?

I am trying to develop a Python based script connected to a SQLite3 database to identify distinct system changepoints in an "online" datastream. Changepoint must be identified in less than 2 minutes ...
4
votes
0answers
666 views

How to estimate a probability distribution

Suppose I want to estimate a probability distribution, is it common practice to simply fit a function to a frequency histogram? So in my work, I am training a classifier, the performance of which is ...
4
votes
1answer
353 views

Does rank of observation matrix tell anything useful when applying machine learning?

Suppose I have an observation matrix of size $N \times M$ where $N$ is the number of samples and $M$ is the number of variables. If the rank of the observation matrix is $R<M$, does it tell ...
4
votes
0answers
1k views

How to draw plot of the values of decision function of multi class svm versus another arbitrary values?

I am trying to draw a plot of the decision function ($f(x)=sign(wx+b)$ which can be obtain by fit$decision.values in R using the svm function of e1071 package) versus another arbitrary values. From ...
4
votes
0answers
450 views

Decomposing the non-deterministic transition functions in non-Markov decision processes into several deterministic transition functions

Problems in reinforcement learning are commonly modeled as Markov decision processes (MDPs). One essential part of MDPs is the transition function $T: S \times A \times S \rightarrow [0, 1] \in \...
4
votes
0answers
242 views

Modelling of probabilistic vs deterministic systems

The learning problem in Statistical Learning Theory is defined as: $$ R(f) = \int_{X,Y} L(y, f(x))P(x,y)\mathrm{d}x\mathrm{d}y $$ where $R(f)$ is the expected risk $L$ is the loss function $P(x, y)...
4
votes
0answers
2k views

Deriving the maximum likelihood for a generative classification model for K classes

In Christopher Bishop's book "Pattern Recognition and Machine learning", there is the following question: Consider a generative classification model for $K$ classes defined by the prior class ...
4
votes
0answers
461 views

Basic idea of zero inflated two part models(hurdel) and application to big data (machine learning)

I'm currently working on the data which has 90% 0s in response variable. Based on my research, it seems zero inflated models could be a solution to this. However, while I was reading related documents,...
4
votes
0answers
1k views

What are appropriate validation methods for a Bayesian network model with low sample size?

I am currently using a Bayesian network model with 20 variables and 210 data points, with 15 locations measured at 14 different time points each. There are also some restrictions on what types of ...
4
votes
0answers
238 views

Duda, Hart, Stork No Free Lunch Discussion

Please see this question regarding Duda, Hart, and Stork's No Free Lunch Theoremm Discussion Hi all, I was having trouble understanding the description of the NFL theorem in Duda, Hart, and Stork. My ...
4
votes
0answers
114 views

Vapniks proof of the basic lemma

In his book Statistical Learning Theory (1998), Vladimir Vapnik proves an inequality needed to prove a bound on the risk for indicator loss functions. Theorem 4.1 on page 133 he derives the following ...
4
votes
0answers
842 views

Learning to Rank: query-dependent vs. query-independent features

I've been doing some reading about learning to rank - specifically lambdaMART - and one thing I am confused about is the role of features. When training a model, should one only use query-dependent ...
4
votes
0answers
162 views

Record linkage when sources have different fields

I have read a little about record linkage, but it seems to me that a requirement is that all fields in both sources can be compared. For example, with sources A and B, an assumption is that we can ...
4
votes
0answers
874 views

Intuition behind perceptron algorithm with offset

I was looking for an intuition for the perceptron algorithm with offset rule, why the update rule is as follows: cycle through all points until convergence $\text{if }\, y^{(t)} \neq \theta^{T}x^{(t)...
4
votes
0answers
366 views

Reducing size of dataset to a fixed size - retaining maximum information in all dimensions

I was wondering about about the following problem: I have a set of $N=10^5$ observations with dimensionality $D=2$, and I would like to reduce it to a set of size with $M=10^3$, or some other $(M \ll ...
4
votes
0answers
3k views

Minimum training sample size required for a classifier

What is the best method to determine the minimum number of training samples required for a classifier? I am only comparing one classifier (four class problem), discriminant function analysis (DFA) ...
4
votes
0answers
2k views

How to train SVM correctly on a 1D dataset

I am trying to use svmtrain (Statistic Toolbox) to train a linear (2 class) SVM on a 1D feature vectors. The features are not fully separable and the classes intersect. The naive approach would be ...
4
votes
0answers
91 views

How to model the distribution of a word game in order to find correlation between demographics and chosen words

I have an experiment (in the form of a word game) whereby people are asked to choose a set of words to describe associations with a topic with the aim of having another person guess the topic. I ...
4
votes
0answers
2k views

SVM classifier (with soft-margin) implementation in R, gamma value and quadprog

I'm trying to implement a Support Vector Machine classifier in R and I have to solve the optimization problem using the quadprog R package which solves problems of the form : $$min_b \frac{1}{2} b^...
4
votes
0answers
559 views

detecting circadian rhythm in a time series

I have a sensor that can detect minute changes in distance. It produces a time series. I would like to point it at people and detect things like their sleeping pattern. How would one build a system ...
4
votes
0answers
625 views

Anomaly detection in user behaviour using hidden Markov models

I would like to detect user anomalies or mal-behavior on a web site. For each user I monitor the web browser used, IP (and thus ISP & geo-location) of the user as well as users' activities on the ...
4
votes
0answers
203 views

Using taxonomic levels as factors in random forests: does it make sense? Is it needed?

I want to test the effect of a set of predictors (ecological and morphological factors) on a categorical response variable (an animal behaviour). As far as I've read, random forests do not make ...
4
votes
0answers
812 views

Efficient Portfolio Optimization Through Simulation

Apologies in advance for the (possibly?) poor terminology as I'm a bit of a novice in the field. I was torn whether to ask this on stackoverflow or here, so hope its the right place. Anyway, my ...
4
votes
0answers
3k views

Adjusting the classification threshold of Naive Bayes

I've been involved in a machine learning project recently and am now in the process of writing the project up for a paper submission. We used the naive bayes classifier on the project and developed a ...
4
votes
1answer
673 views

Formula for marginal probability in CRF++

On the website for CRF++ http://crfpp.sourceforge.net/ they mention that marginal probabilities can be output for each possible label. My question is, in CRF theory, what's the formula for this ...
4
votes
1answer
408 views

Observation symbols for training a set of HMMs

If we are to classify 2 separate classes/actions using HMMs, we design 2 separate HMMs (one for each class). Do they share same set or a different set of observations-symbols for each of the HMM? If ...
4
votes
0answers
2k views

Why would concatenating feature vectors lead to better estimates?

I wish to estimate the state of a system from two separate and disparate observations. A simple approach that I have seen in some literature is to combine the feature vectors (observations) by simply ...
4
votes
0answers
310 views

Category selection for text classification

It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
4
votes
2answers
110 views

Practical realities of updating a trained model with new data

In my day to day work, I train models on data using R packages that have no extension for Bayesian priors. I will generally have a large dataset to start off with, and add new data as needed. Any ...
4
votes
2answers
464 views

Unlearning Neural Network? Prevent learning from a specific feature

Is it possible to train a NN to avoid the features that a different neural network finds? For example, let's train a simple 1 layer CNN with 1x1 kernels on a supervised binary classification problem. ...