Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

4,389 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
5
votes
1answer
363 views

Unsupervised pre-training for Reinforcement Learning

Since the advent of many unsupervised learning methods, as a pretraining step for the main supervised task (mostly under the name of Deep Learning), it shouldn't be strange to ask, what is the current ...
5
votes
0answers
419 views

Any implementations of fully recurrent neural networks applied to reinforcement learning?

I've seen a single paper on the topic of adapting fully recurrent networks to a reinforcement learning setting, but according to google scholar its had no citations and no code has been released ...
5
votes
0answers
1k views

k-fold cross validation vs k times hold-out validation

I am facing the evaluation of a genetic programming algorithm. I am using the Proben1 cancer1 dataset to evaluate the models created by this algorithm. This dataset contains 699 samples, which is ...
5
votes
0answers
179 views

How to learn similarity of typed/attributed graphs?

I have a question for graph machine learning gurus :). For this project I'm working on, I need to be able to learn similarity between typed graphs. By typed I mean that every vertex and every edge of ...
5
votes
0answers
121 views

Graphical nominal model

Suppose I have a set of $k$ matrices. $$ \mathbb D = A_1,A_2,...,A_k $$ Each column of $A$ is categorical vector. $$ A = v_1,v_2,...,v_n $$ I want to find the mapping $$ f: A \...
5
votes
0answers
175 views

Asynchronous data stream matching

Suppose you have a classifier $C^n$ which continuously outputs a stream of classification labels $K^n_i$ and corresponding timestamps $T^n_i$. Also, we know the prior probability $P(K^n) \forall n$. ...
4
votes
0answers
83 views

How to model probabilistic inputs with continuous output using regression

I have trained a multi-output classifier that takes an image as input and returns softmax logits as output. To be specific, the multi-output classifier takes an image and says the probability that ...
4
votes
1answer
111 views

Time Series Regressor Selection

I am interest in a (multivariate) algorithm to identify relevant regressors (which are itself time series) to forecast a time series of interest. The question is worded in general terms because this ...
4
votes
0answers
43 views

How to reconstruct an image from a training set?

Description: I have taken a series of images/photos of a panorama from different positions (x,y) in space pretty close to each other (max 100m difference). Here there is a top view representation to ...
4
votes
0answers
24 views

How to combine noisy and noise-free datasets to train a model

Overview Suppose I have two datasets, both of which consist of rows of features and their matching labels. One of these datasets is noise-free and its labels correspond to the ground truth, but the ...
4
votes
0answers
75 views

Why is this statistic F-distributed?

A book I'm reading claims that the statistic: $\frac{(RSS_0 - RSS_1) / (p_1 - p_0)}{RSS_1 / (N - p_1 - 1)}$ has an F distribution. Why is this? I know that an F distribution is something like $\frac{\...
4
votes
0answers
166 views

PCA, SMOTE and cross validation - how to combine them together?

I was reading a lot recently about PCA and cross validation and it seems that the majority call it malpractice to do PCA before cross validation. I would also like to perform SMOTE, but there is a ...
4
votes
0answers
172 views

what is the mistake of convergence proof in Adam

Sashank J. Reddi et. al in their paper "On the convergence of Adam and beyond" say that, Adam's proof of convergence as stated in original paper is wrong. More than that, they point out that the value ...
4
votes
0answers
54 views

Inferring a Markov chain from its invariant measure

Given a probability measure $p$ on $\{1,\dots,n\}$ assumed to be the invariant measure of some irreducible ergodic Markov chain with unknown transition matrix $P$, i.e., $p = pP$, what (if any) ...
4
votes
0answers
202 views

Correcting Sample Selection Bias given actual Distribution

I have two datasets, both from the same population: The samples from the first survey are quite representative of the underlying truth. However, the second survey comes with a change in distribution ...
4
votes
0answers
46 views

soft SVM - degenerate case

According to "A Note on Support Vector Machine Degeneracy", Theorem 4, if the dual problem for soft-SVM has a solution with $\alpha_i \in \{0,C\}, \forall i$, then $w=0$ for the primal problem. In "...
4
votes
1answer
149 views

Neural Networks - Strategies for problems with high Bayes error rate

I am building a Neural Network for a binary classification problem where the Bayes error (lowest possible error rate) is probably close to 50%. What makes the task easier is that I don't need to make ...
4
votes
1answer
50 views

Gradual clustering in deterministic manner

We have 128-dimensional vectors representing people's identities where the euclidean metric defines the similarity between them. Ours solution requires them to be clustered and then annotated (assign ...
4
votes
1answer
36 views

How do these matrices form an order-$4$-tensor?

I'm reading this paper on a convolutional neural network for modelling sentences, and I'm having some trouble understanding section $3.5$. Please consider the following text: We denote a feature map ...
4
votes
0answers
163 views

Why do we use gradients instead of residuals in Gradient Boosting?

I have found mentions of two advantages in using gradients instead of actual residuals: 1) Using gradients will allow us to plug in any loss function (not just mse) without having to change our base ...
4
votes
0answers
113 views

A few questions regarding the practice of heterogeneous treatment effect analysis (a.k.a, interaction detection or subgroup analysis) methods

Imagine I am looking at a randomized experiment between a control and one or more treatment conditions. For example, I have a treatment that aims to get people out of debt. I randomize people to ...
4
votes
0answers
57 views

How does one design a data set from polynomial target function such that logistic regression separates the data perfectly?

I want to design a target function for a classification task of the form: $$ f_{target}(x) = \mathbb{1}_{>0}[\sum^{D^*}_{i=0} w^*_i x^i] = \mathbb{1}_{>0}[ \langle w^*, \Phi(x)\rangle]$$ and ...
4
votes
0answers
484 views

Effectiveness of Standardization and Normalization in Machine Learning

I am just studying the basics of machine learning and had a question about the standardisation and normalisation of the features and its effectiveness. I have read this CrossValidated question and ...
4
votes
0answers
41 views

using sparse models to accelerate training

I was going through a tutorial that highlights the importance of using sparse models to get "lightning fast models" i.e. to accelerate the training process. What are the issues in using a sparse ...
4
votes
0answers
607 views

Supervised Learning: Cosine Similarity as Loss function?

For supervised learning, the loss function should be differentiable so that back-propagation can be performed. I am wondering if it is possible to use loss function that computes the cosine similarity?...
4
votes
0answers
246 views

Adversarial examples - regularization method

In Intriguing properties of neural networks (https://arxiv.org/pdf/1312.6199.pdf) they show (4.3), that the existance of adversarial examples is closely connected to the upper Lipschitz constant, ...
4
votes
1answer
347 views

General procedures for combined feature selection, model tuning, and model selection?

What is the general procedure for a combined task of model tuning (i.e., hyperparameter selection), feature selection and model selection? I know some basic principles for each task, but when ...
4
votes
0answers
684 views

How to include negative examples in multi-class classification?

I have a problem similar to this question: How do I use negative examples (in addition to positive ones) for training a multiclass softmax classifier (or a neural net with softmax output)? where I ...
4
votes
0answers
37 views

Algorithm that preserves the order of the predicted variable

Hi all, I need some advice on possible algorithms that I can apply to the following problem (if possible with pointers to implementations of these algorithms). The dataset: I have some dataset ...
4
votes
0answers
2k views

How does XGBoost/lightGBM evaluate ndcg metric for ranking

I am currently running tests between XGBoost/lightGBM for their ability to rank items. I am reproducing the benchmarks presented here: https://github.com/guolinke/boosting_tree_benchmarks. I have ...
4
votes
1answer
293 views

Is that possible to distill the knowledge of a stacked ensemble model?

There is a famous paper "distilling the knowledge in a neural network" from Hinton about training a small NN to represent a large deep NN. Is that possible to do the same thing for a stacked Ensemble ...
4
votes
1answer
1k views

Condition for RNN vanishing gradients and eigenvalues of the matrix of weights

In this article on recurrent neural networks by Razvan Pascanu, $\mathbf x_t$ is the state at time $t;$ $\mathbf u_t$ the input at time $t$; and $\mathcal E$ is the cost function: A proof is given of ...
4
votes
0answers
41 views

What type of model can be used to detect changes in periodic behavior?

Imagine we have a data sequence centered around 0 with small fluctuations +/- 1, but approximately every 100 observations it jumps to 10. If this behavior changed and it started jumping to 5 every 50 ...
4
votes
1answer
372 views

Modeling delayed feedback using logistic regression

Suppose we are trying to model the probability of a user clicking on an ad using logistic regression. We will receive only the positive feedback so, we define $Y = 1$ when success was observed, $Y=0$ ...
4
votes
0answers
211 views

Machine Learning for Causal Inference with Panel Data: Possible to combine ML estimators with additive/linear terms to derive diff-in-diff estimator?

My question is motivated by the following. First consider the non-panel case, where we have two groups, the treated group ($g=t$) and the comparison group ($g=c$), and are trying to estimate an ...
4
votes
1answer
656 views

Regression Trees' greedy algorithm in Hastie et al. (2009)

I am reading The Elements of Statistical Learning (Hastie, Tibshirani and Friedman, 2009), more specifically the section on regression decision trees (p. 307 of the book). There is something I do not ...
4
votes
0answers
245 views

Using holdout cross validation in the inner loop of nested cross validation

Is it valid to do a k-fold CV in the outer loop of a nested CV but use the holdout method in the inner loop to avoid computational complexity? My guess is that one could leave one of the k sets out ...
4
votes
0answers
241 views

classification: concatenating descriptors vs. using multiple classifiers

Consider a typical machine learning problem where you try to do object classification from a high-dimensional set of features. Suppose we know that the features are actually a collection of distinct "...
4
votes
0answers
201 views

Machine learning on short time series with multiple observations at one point in time

I have a time series which is short i.e it has only 7 points (7 months) where the data is measured(Margin), it has many multi level categorical attributes and one numerical attribute called ...
4
votes
0answers
181 views

Is one-vs-all logit or multionomial logit regression more accurate?

What is advice of when to use one-vs-all logit or multinomial logit regressions? Most importantly, which one has a higher prediction power? Can one test hypothesis and estimate confidence intervals in ...
4
votes
0answers
789 views

Why is the derivative of the LSTM cell state w.r.t. to the previous cell state equal to the forget gate?

I keep seeing this online, on Quora and Machine Learning subreddits but I don't get it. Here's some basic math to show otherwise: We use this equation for the cell state: $c_t = f_t \odot c_t\__1 + ...
4
votes
0answers
305 views

Google gender-pay gap vs

Background: I read this: google schools US government about gender pay gap. It derives from this google blog post by Eileen Naughton, VP of People Operations. She asserts that google is somehow "...
4
votes
1answer
275 views

Measuring the bias-variance tradeoff

Does anyone know of a metric that quantifies the bias-variance tradeoff of a given fitted model? I'm not talking about measuring the MSE in cross validation, I'm interested in a single generic or ...
4
votes
0answers
110 views

What is the feasibility to run linear model for large amount of data?

I am looking for empirical approximations and guidelines on how many operations are required to run a linear model on a given amount of data, or if it's even feasible. Let's assume we use QR ...
4
votes
0answers
328 views

Using Char RNN to Capitalize Text

As this paper (http://www.statnlp.org/research/ta/rnn_truecase.pdf) illustrates, a char rnn can be used to true-case some text i..e capitalize the necessary characters. The paper just says it uses a ...
4
votes
0answers
414 views

How to check t-SNE performance

We use t-SNE to visualize multidimensional vectors into 2D or 3D space, and if my understanding of this algorithm is correct, we can compare it to PCA as it also provides dimensionality reduction. My ...
4
votes
1answer
589 views

the role of basis functions in reinforcement learning

In the very simple examples of reinforcement learning (gridworld, mountain car), we use real numbers or some elementary functions as reward functions. When state spaces become larger and larger, and ...
4
votes
1answer
59 views

In variational inference on von Mises clusters, how to find a bound for the log-Bessel function?

This paper on von Mises clustering uses an upper bound on the modified log-Bessel function that I struggle to replicate. Taking results from this paper, the authors state: $$u\frac{I'_\nu(u)}{I_\nu(u)...
4
votes
0answers
397 views

Train Neural Network For Handwritten Chinese Characters

The article here: http://novanoid.github.io/2014/09/26/training-a-neural-network-to-recognize-handwritten-digits/ discusses and implements a way to recognize handwritten digits. For images with a ...
4
votes
0answers
219 views

Ensemble learning with time-varying covariates and effects

We are interested in replicating several duration studies in the literature using ensemble learning methods. After some experimentation, we opted for random survival forests (Ishwaran et al. 2008) for ...