Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

4
votes
0answers
56 views

How does one design a data set from polynomial target function such that logistic regression separates the data perfectly?

I want to design a target function for a classification task of the form: $$ f_{target}(x) = \mathbb{1}_{>0}[\sum^{D^*}_{i=0} w^*_i x^i] = \mathbb{1}_{>0}[ \langle w^*, \Phi(x)\rangle]$$ and ...
4
votes
0answers
460 views

Effectiveness of Standardization and Normalization in Machine Learning

I am just studying the basics of machine learning and had a question about the standardisation and normalisation of the features and its effectiveness. I have read this CrossValidated question and ...
4
votes
0answers
36 views

using sparse models to accelerate training

I was going through a tutorial that highlights the importance of using sparse models to get "lightning fast models" i.e. to accelerate the training process. What are the issues in using a sparse ...
4
votes
0answers
574 views

Supervised Learning: Cosine Similarity as Loss function?

For supervised learning, the loss function should be differentiable so that back-propagation can be performed. I am wondering if it is possible to use loss function that computes the cosine similarity?...
4
votes
0answers
234 views

Adversarial examples - regularization method

In Intriguing properties of neural networks (https://arxiv.org/pdf/1312.6199.pdf) they show (4.3), that the existance of adversarial examples is closely connected to the upper Lipschitz constant, ...
4
votes
0answers
623 views

How to include negative examples in multi-class classification?

I have a problem similar to this question: How do I use negative examples (in addition to positive ones) for training a multiclass softmax classifier (or a neural net with softmax output)? where I ...
4
votes
0answers
34 views

Algorithm that preserves the order of the predicted variable

Hi all, I need some advice on possible algorithms that I can apply to the following problem (if possible with pointers to implementations of these algorithms). The dataset: I have some dataset ...
4
votes
0answers
2k views

How does XGBoost/lightGBM evaluate ndcg metric for ranking

I am currently running tests between XGBoost/lightGBM for their ability to rank items. I am reproducing the benchmarks presented here: https://github.com/guolinke/boosting_tree_benchmarks. I have ...
4
votes
0answers
41 views

What type of model can be used to detect changes in periodic behavior?

Imagine we have a data sequence centered around 0 with small fluctuations +/- 1, but approximately every 100 observations it jumps to 10. If this behavior changed and it started jumping to 5 every 50 ...
4
votes
0answers
202 views

Machine Learning for Causal Inference with Panel Data: Possible to combine ML estimators with additive/linear terms to derive diff-in-diff estimator?

My question is motivated by the following. First consider the non-panel case, where we have two groups, the treated group ($g=t$) and the comparison group ($g=c$), and are trying to estimate an ...
4
votes
0answers
224 views

Using holdout cross validation in the inner loop of nested cross validation

Is it valid to do a k-fold CV in the outer loop of a nested CV but use the holdout method in the inner loop to avoid computational complexity? My guess is that one could leave one of the k sets out ...
4
votes
0answers
231 views

classification: concatenating descriptors vs. using multiple classifiers

Consider a typical machine learning problem where you try to do object classification from a high-dimensional set of features. Suppose we know that the features are actually a collection of distinct "...
4
votes
0answers
173 views

Machine learning on short time series with multiple observations at one point in time

I have a time series which is short i.e it has only 7 points (7 months) where the data is measured(Margin), it has many multi level categorical attributes and one numerical attribute called ...
4
votes
0answers
176 views

Is one-vs-all logit or multionomial logit regression more accurate?

What is advice of when to use one-vs-all logit or multinomial logit regressions? Most importantly, which one has a higher prediction power? Can one test hypothesis and estimate confidence intervals in ...
4
votes
0answers
305 views

Google gender-pay gap vs

Background: I read this: google schools US government about gender pay gap. It derives from this google blog post by Eileen Naughton, VP of People Operations. She asserts that google is somehow "...
4
votes
0answers
107 views

What is the feasibility to run linear model for large amount of data?

I am looking for empirical approximations and guidelines on how many operations are required to run a linear model on a given amount of data, or if it's even feasible. Let's assume we use QR ...
4
votes
0answers
312 views

Using Char RNN to Capitalize Text

As this paper (http://www.statnlp.org/research/ta/rnn_truecase.pdf) illustrates, a char rnn can be used to true-case some text i..e capitalize the necessary characters. The paper just says it uses a ...
4
votes
0answers
382 views

How to check t-SNE performance

We use t-SNE to visualize multidimensional vectors into 2D or 3D space, and if my understanding of this algorithm is correct, we can compare it to PCA as it also provides dimensionality reduction. My ...
4
votes
0answers
381 views

Train Neural Network For Handwritten Chinese Characters

The article here: http://novanoid.github.io/2014/09/26/training-a-neural-network-to-recognize-handwritten-digits/ discusses and implements a way to recognize handwritten digits. For images with a ...
4
votes
0answers
196 views

Ensemble learning with time-varying covariates and effects

We are interested in replicating several duration studies in the literature using ensemble learning methods. After some experimentation, we opted for random survival forests (Ishwaran et al. 2008) for ...
4
votes
0answers
174 views

How do you evaluate machine learning model already deployed in production?

so to be more clear lets consider the problem of loan default prediction. Let's say I have trained and tested off-line multiple classifiers and ensembled them. Then I gave this model to production. ...
4
votes
0answers
279 views

Unsupervised Anomaly Detection Threshold Selection

If we have a data set that contains only positive examples I am wondering how we can effectively choose a threshold for an anomaly detection technique. Are there anomaly detection techniques that can ...
4
votes
0answers
142 views

Sparse Representation, Sparse Learning, Sparse Coding, Group Sparse Coding and Group Sparse Learning?

I'm really confused with these terms for the relations and difference between them: Sparse Representation Sparse Learning Sparse Coding Group Sparse Coding Group Sparse Learning Sparse Dictionary ...
4
votes
0answers
468 views

How does a maze game / directed paths map to a Neural Network?

I am trying to understand how a maze game's possible paths and possible moves are mapped out in a Neural Network. Let's take this example here: http://cdn.intechopen.com/pdfs-wm/10916.pdf Agent can ...
4
votes
0answers
95 views

Can I use HMM to predict the spread of Ebola?

1) Can Hidden Markov Model be used across both a large number of categories (districts) and cases (weeks)? 2) Is HMM appropriate for trying to model such a problem? 3) Would I need to develop a ...
4
votes
0answers
79 views

Training instances importance in Random Forest?

Is it possible to determine the importance of the training examples in Random Forests, analogously to what's done with predictors? Basically the idea would be to find important samples in the data, ...
4
votes
0answers
7k views

Splitting data for train/test for time series

A week ago or so I was at a conference. Long story short, I ran into a friend who is quite good at machine learning so I asked them a question about why I might be getting what I think is poor fit on ...
4
votes
0answers
139 views

How to mitigate the hierarchical error propagation in tree-structured classification

Suppose we have a multi-class classification problem, where the number of classes $K \geq 3$ We use a tree structure of multiple SVMs to divide and conquer the problem, with one example in the figure ...
4
votes
0answers
662 views

Can SMOTE deal with categorical variables

I am dealing with a highly unbalanced dataset to detect hacker intrusion in a network (~23k 0 or negative events, and ~1k 1 or positive events). I am using sklearn random forest algorithm. To deal ...
4
votes
0answers
3k views

Bagging of xgboost

The extreme-gradient boosting algorithm seems to be widely applied these days. I often have the feeling that boosted models tend to overfit. I know that there are parameters in the algorithm to ...
4
votes
0answers
574 views

How to estimate a probability distribution

Suppose I want to estimate a probability distribution, is it common practice to simply fit a function to a frequency histogram? So in my work, I am training a classifier, the performance of which is ...
4
votes
0answers
1k views

How to draw plot of the values of decision function of multi class svm versus another arbitrary values?

I am trying to draw a plot of the decision function ($f(x)=sign(wx+b)$ which can be obtain by fit$decision.values in R using the svm function of e1071 package) versus another arbitrary values. From ...
4
votes
0answers
440 views

Decomposing the non-deterministic transition functions in non-Markov decision processes into several deterministic transition functions

Problems in reinforcement learning are commonly modeled as Markov decision processes (MDPs). One essential part of MDPs is the transition function $T: S \times A \times S \rightarrow [0, 1] \in \...
4
votes
0answers
239 views

Modelling of probabilistic vs deterministic systems

The learning problem in Statistical Learning Theory is defined as: $$ R(f) = \int_{X,Y} L(y, f(x))P(x,y)\mathrm{d}x\mathrm{d}y $$ where $R(f)$ is the expected risk $L$ is the loss function $P(x, y)...
4
votes
0answers
2k views

Deriving the maximum likelihood for a generative classification model for K classes

In Christopher Bishop's book "Pattern Recognition and Machine learning", there is the following question: Consider a generative classification model for $K$ classes defined by the prior class ...
4
votes
0answers
446 views

Basic idea of zero inflated two part models(hurdel) and application to big data (machine learning)

I'm currently working on the data which has 90% 0s in response variable. Based on my research, it seems zero inflated models could be a solution to this. However, while I was reading related documents,...
4
votes
0answers
1k views

What are appropriate validation methods for a Bayesian network model with low sample size?

I am currently using a Bayesian network model with 20 variables and 210 data points, with 15 locations measured at 14 different time points each. There are also some restrictions on what types of ...
4
votes
0answers
233 views

Duda, Hart, Stork No Free Lunch Discussion

Please see this question regarding Duda, Hart, and Stork's No Free Lunch Theoremm Discussion Hi all, I was having trouble understanding the description of the NFL theorem in Duda, Hart, and Stork. My ...
4
votes
0answers
109 views

Vapniks proof of the basic lemma

In his book Statistical Learning Theory (1998), Vladimir Vapnik proves an inequality needed to prove a bound on the risk for indicator loss functions. Theorem 4.1 on page 133 he derives the following ...
4
votes
0answers
821 views

Learning to Rank: query-dependent vs. query-independent features

I've been doing some reading about learning to rank - specifically lambdaMART - and one thing I am confused about is the role of features. When training a model, should one only use query-dependent ...
4
votes
0answers
152 views

Record linkage when sources have different fields

I have read a little about record linkage, but it seems to me that a requirement is that all fields in both sources can be compared. For example, with sources A and B, an assumption is that we can ...
4
votes
0answers
803 views

Intuition behind perceptron algorithm with offset

I was looking for an intuition for the perceptron algorithm with offset rule, why the update rule is as follows: cycle through all points until convergence $\text{if }\, y^{(t)} \neq \theta^{T}x^{(t)...
4
votes
0answers
3k views

Minimum training sample size required for a classifier

What is the best method to determine the minimum number of training samples required for a classifier? I am only comparing one classifier (four class problem), discriminant function analysis (DFA) ...
4
votes
0answers
2k views

How to train SVM correctly on a 1D dataset

I am trying to use svmtrain (Statistic Toolbox) to train a linear (2 class) SVM on a 1D feature vectors. The features are not fully separable and the classes intersect. The naive approach would be ...
4
votes
0answers
91 views

How to model the distribution of a word game in order to find correlation between demographics and chosen words

I have an experiment (in the form of a word game) whereby people are asked to choose a set of words to describe associations with a topic with the aim of having another person guess the topic. I ...
4
votes
0answers
2k views

SVM classifier (with soft-margin) implementation in R, gamma value and quadprog

I'm trying to implement a Support Vector Machine classifier in R and I have to solve the optimization problem using the quadprog R package which solves problems of the form : $$min_b \frac{1}{2} b^...
4
votes
0answers
534 views

detecting circadian rhythm in a time series

I have a sensor that can detect minute changes in distance. It produces a time series. I would like to point it at people and detect things like their sleeping pattern. How would one build a system ...
4
votes
0answers
598 views

Anomaly detection in user behaviour using hidden Markov models

I would like to detect user anomalies or mal-behavior on a web site. For each user I monitor the web browser used, IP (and thus ISP & geo-location) of the user as well as users' activities on the ...
4
votes
0answers
200 views

Using taxonomic levels as factors in random forests: does it make sense? Is it needed?

I want to test the effect of a set of predictors (ecological and morphological factors) on a categorical response variable (an animal behaviour). As far as I've read, random forests do not make ...
4
votes
0answers
794 views

Efficient Portfolio Optimization Through Simulation

Apologies in advance for the (possibly?) poor terminology as I'm a bit of a novice in the field. I was torn whether to ask this on stackoverflow or here, so hope its the right place. Anyway, my ...