Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [machine-learning]

Methods and principles of building "computer systems that try to automatically improve with experience."

28
votes
0answers
1k views

Link Anomaly Detection in Temporal Network

I came across this paper that uses link anomaly detection to predict trending topics, and I found it incredibly intriguing: The paper is "Discovering Emerging Topics in Social Streams via Link Anomaly ...
17
votes
0answers
618 views

How does a Relevance Vector Machine (RVM) work?

Relevance Vector Machines (RVMs) are really interesting models when contrasted with the highly geometrical (and popular) SVMs. In the light of a question like How does a Support Vector Machine (SVM) ...
12
votes
0answers
667 views

Bayesian Q-learning

Suppose that, for every state $s$, there is a set of actions $\mathcal{A}(s)$ that can be chosen in that state. Let $Q(s, a)$ denote the expected utility of choosing action $a \in \mathcal{A}(s)$ in ...
11
votes
0answers
896 views

Rademacher complexity of logistic regression

Consider logistic regression. We have the logistic loss function, $\phi: R\rightarrow [0,1], \phi(u)=\log(1+\exp(-u))$, which is Lipschitz, and we have the linear function class $F=\{f_w:R^d \...
9
votes
0answers
409 views

In Random Forest, why is a random subset of features chosen at the node level rather than at the tree level?

My Question: Why does random forest consider random subsets of features for splitting at the node level within each tree rather than at the tree level? Background: This is something of a history ...
8
votes
0answers
2k views

Backpropagation on a convolutional layer

Online tutorials describe in depth the convolution of an image with a filter, etc; However, I have not seen one that describes the backpropagation on the filter (at least visually). First let me try ...
8
votes
0answers
338 views

Can Frank Harrell's method be used to obtain optimism-corrected regression coefficients?

I used a regularized (LASSO) Cox regression to estimate relapse times of patients and used Frank Harrell's bootstrapping method to obtain an optimism-corrected performance estimate of my model. ...
8
votes
0answers
6k views

Interpreting negative cosine similarity

My question may be a silly one. So I shall apologize in advance. I was trying to use the GLOVE model pre-trained by Stanford NLP group (link). However, I noticed that my similarity results showed ...
7
votes
0answers
54 views

Do Stochastic Processes such as the Gaussian Process/Dirichlet Process have densities? If not, how can Bayes rule be applied to them?

The Dirichlet Pocess and Gaussian Process are often referred to as "distributions over functions" or "distributions over distributions". In that case, can I meaningfully talk about the density of a ...
7
votes
0answers
93 views

Why does entropy as error measure leads to overfitting?

This post on KDnuggets quoting the post by John Langford states that entropy and mutual information when used as error measures can lead to overfitting. Could you go into more details on this?
7
votes
0answers
286 views

Reinforcement *Model* Learning

Classical reinforcement learning (Q- or Sarsa-Learning) can be extended with models of the environment. These models are usually transition tables that contain the probability of arriving at a ...
7
votes
0answers
518 views

Machine learning with ordered labels

The usual method for adapting binary classifiers like various SVMs to multilabel data is one-vs-all, which assumes that labels are independent and in case of a prediction error we don't care what ...
7
votes
0answers
1k views

Post processing random forests using regularised regression: what about bias?

I have been playing around with post processing the results of the random forest for regression machine learning algorithm in order to try and do better than the default mean of all trees prediction. ...
6
votes
0answers
415 views

Compatible Function Approximation Theorem in Reinforcement Learning

In the Compatible Function Approximation Theorem, the following condition is required to make the policy gradient to be exact $\nabla J(\theta) = \mathbb{E}_{\pi_{\theta}}\left [\nabla_{\theta}log\pi_{...
6
votes
0answers
295 views

Inter-rater agreement of a gold standard dataset - a ceiling for reliable evaluation of algorithms?

In my field, a dated gold standard dataset is used to track progress in algorithm development. Now when the state-of-the-art algorithms obtain higher correlation than is the inter-rater agreement of ...
6
votes
0answers
1k views

How can I get feature importance for Gaussian Naive Bayes classifier?

I have a dataset consisting of 4 classes and around 200 features. I have implemented a Gaussian Naive Bayes classifier. I want now calculate the importance of each feature for each pair of classes ...
6
votes
0answers
746 views

Classification of multiple time series and case level attributes

I'm pretty new to machine learning so wondering whether someone can help check my thinking or point me in the right direction! I need to create a classifier which can predict an outcome for a person ...
6
votes
0answers
2k views

Maximum entropy classifier and sentiment analysis

I am doing a project work in sentiment analysis (on Twitter data) using machine learning approach. In order to find the 'best' way to this I have experimented with naive Bayesian and maximum entropy ...
5
votes
0answers
151 views

Quantifying uncertainty when fitting a statistical model to partial effects/dependencies from a random forest (or other machine learning model)

Question: I estimate the partial dependence of $y$ on one predictor in a fitted random forest (RF). I want to now fit a parametric model to this partial dependence. How can I estimate my uncertainty ...
5
votes
0answers
97 views

Elastic net arbitrary alpha selection

I'm trying to solve a prediction problem given the following constraints: I need an interpretable model to be used for experimental validation I need a model that performs feature selection to reduce ...
5
votes
0answers
126 views

When is there a free lunch?

The no free lunch theorem (NFL) states that Theorem (Wolpert and Macready 1997) Let $A$ be any learning algorithm for the task of binary classification with respect to the $0−1$ loss over a ...
5
votes
0answers
537 views

Applying machine learning techniques to panel data

I have a panel data in which I observe 1500 companies and many individuals work for those companies for multiple periods. I have explanatory variables at both individual (e.g. race, age, education) ...
5
votes
0answers
4k views

Change image input size of a pre-trained convnet

maybe this question will sound a bit as a newbie one but I'd like to have some clarification. I'm using a VGG16-like convnet, pre-trained with VGG16 weights and edited top layers to work with my ...
5
votes
0answers
2k views

Deriving linear regression gradient with MSE

So I've been tinkering around with the backpropagation algorithm and to try to get a better understanding of how it works and my calculus is quite rusty. I've derived the gradient for linear ...
5
votes
0answers
286 views

How do bias, variance and overfitting relate to each other?

I'm quite new to Machine Learning, and after reading about the bias-variance tradeoff and overfitting/underfitting, several questions raised in my mind: If I have a model with 15% error on train set ...
5
votes
0answers
243 views

What good are Rademacher bounds?

VC complexity bounds are generally well-known for common hypothesis classes based on supervised learning ERM algorithms. However, VC bounds are independent of the distribution from which the data is ...
5
votes
0answers
187 views

Using holdout cross validation in the inner loop of nested cross validation

Is it valid to do a k-fold CV in the outer loop of a nested CV but use the holdout method in the inner loop to avoid computational complexity? My guess is that one could leave one of the k sets out ...
5
votes
0answers
296 views

Google gender-pay gap vs

Background: I read this: google schools US government about gender pay gap. It derives from this google blog post by Eileen Naughton, VP of People Operations. She asserts that google is somehow "...
5
votes
0answers
197 views

What is the utility/significance of PAC learnability and VC dimension?

I've been reading Shalev-Shwartz & Ben-David's book, "Understanding Machine Learning", which presents the PAC theory in its Part I. While the theory of PAC learnability does appear very elegant ...
5
votes
0answers
446 views

Deep Learning vs Structured Learning

I am interested in the differences between using large, deep learning networks vs Probabilistic graphical models (PGMs), like Random Field models, for structured learning (e.g. on images, or labels of ...
5
votes
0answers
6k views

Splitting data for train/test for time series

A week ago or so I was at a conference. Long story short, I ran into a friend who is quite good at machine learning so I asked them a question about why I might be getting what I think is poor fit on ...
5
votes
0answers
219 views

Is there an appropriate order to apply bagging and filter feature selection?

I'm training a (regression) learner on a $p \gg n$ problem, including bagging and filter feature selection (information gain). I'm in doubt though regarding the order of the procedures: Apply the ...
5
votes
0answers
7k views

True positive, false negative, true negative, false positive definitions for multiclass-multilabel classification?

I'm trying to apply some evaluation metrics to several clustering methods. I thought that I knew them basing on the multiclass confusion matrix, considering the rows as the actual classes and the ...
5
votes
0answers
1k views

Dealing with auxiliary random variables for Mean-Field Variational Inference in Bayesian Poisson factorization

I am studying as a part of a class assignment a recent paper on Poisson factorization. Some points of the paper regarding the usage of some auxiliary variables are not clear to me. I would like to ...
5
votes
0answers
1k views

Recurrent neural network for object tracking & position filtering?

Would a recurrent neural network be appropriate for object tracking tasks? Mainly I will have 3D feature vectors $(x, y, t)$ where $x$ and $y$ are the positions of an object in the image and $t$ is ...
5
votes
0answers
280 views

Interpreting hidden layer representations in ANNs

I'm using the fann library for writing an Artificial Neural Network in C++. I trained my network for the task of recognizing faces inside a set of 128x128 .png images, using three different algorithms:...
5
votes
0answers
390 views

Ideal statistical or machine learning technique to model highly cross-correlated data

I'm trying to build a model that can predict streamflow for an alpine (snowmelt-fed) watershed using snow albedo (roughly, the energy reflectance of the snow) data. Albedo controls the melt of the ...
5
votes
0answers
147 views

Compressed sensing: Optimization in $L_1$ norm and total variation with fourier coefficients

I'm reading the article Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information (Candes, Romberg and Tao, 2004). In this article they are talking ...
5
votes
0answers
165 views

How to learn similarity of typed/attributed graphs?

I have a question for graph machine learning gurus :). For this project I'm working on, I need to be able to learn similarity between typed graphs. By typed I mean that every vertex and every edge of ...
5
votes
0answers
118 views

Graphical nominal model

Suppose I have a set of $k$ matrices. $$ \mathbb D = A_1,A_2,...,A_k $$ Each column of $A$ is categorical vector. $$ A = v_1,v_2,...,v_n $$ I want to find the mapping $$ f: A \...
5
votes
0answers
167 views

Asynchronous data stream matching

Suppose you have a classifier $C^n$ which continuously outputs a stream of classification labels $K^n_i$ and corresponding timestamps $T^n_i$. Also, we know the prior probability $P(K^n) \forall n$. ...
4
votes
0answers
38 views

“Hierarchical” Random forests?

Background I am using Random Forest to classify ~900 objects based on a large number (> 80) predictors. I split these 70:30 for training and testing. The overall model does fairly well, giving an ...
4
votes
0answers
85 views

what is the mistake of convergence proof in Adam

Sashank J. Reddi et. al in their paper "On the convergence of Adam and beyond" say that, Adam's proof of convergence as stated in original paper is wrong. More than that, they point out that the value ...
4
votes
0answers
159 views

Correcting Sample Selection Bias given actual Distribution

I have two datasets, both from the same population: The samples from the first survey are quite representative of the underlying truth. However, the second survey comes with a change in distribution ...
4
votes
0answers
35 views

soft SVM - degenerate case

According to "A Note on Support Vector Machine Degeneracy", Theorem 4, if the dual problem for soft-SVM has a solution with $\alpha_i \in \{0,C\}, \forall i$, then $w=0$ for the primal problem. In "...
4
votes
0answers
124 views

Why do we use gradients instead of residuals in Gradient Boosting?

I have found mentions of two advantages in using gradients instead of actual residuals: 1) Using gradients will allow us to plug in any loss function (not just mse) without having to change our base ...
4
votes
0answers
50 views

How does one design a data set from polynomial target function such that logistic regression separates the data perfectly?

I want to design a target function for a classification task of the form: $$ f_{target}(x) = \mathbb{1}_{>0}[\sum^{D^*}_{i=0} w^*_i x^i] = \mathbb{1}_{>0}[ \langle w^*, \Phi(x)\rangle]$$ and ...
4
votes
0answers
394 views

Effectiveness of Standardization and Normalization in Machine Learning

I am just studying the basics of machine learning and had a question about the standardisation and normalisation of the features and its effectiveness. I have read this CrossValidated question and ...
4
votes
0answers
25 views

using sparse models to accelerate training

I was going through a tutorial that highlights the importance of using sparse models to get "lightning fast models" i.e. to accelerate the training process. What are the issues in using a sparse ...
4
votes
0answers
496 views

Supervised Learning: Cosine Similarity as Loss function?

For supervised learning, the loss function should be differentiable so that back-propagation can be performed. I am wondering if it is possible to use loss function that computes the cosine similarity?...