Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

4,268 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
30
votes
0answers
1k views

Link Anomaly Detection in Temporal Network

I came across this paper that uses link anomaly detection to predict trending topics, and I found it incredibly intriguing: The paper is "Discovering Emerging Topics in Social Streams via Link Anomaly ...
18
votes
0answers
753 views

How does a Relevance Vector Machine (RVM) work?

Relevance Vector Machines (RVMs) are really interesting models when contrasted with the highly geometrical (and popular) SVMs. In the light of a question like How does a Support Vector Machine (SVM) ...
12
votes
0answers
748 views

Bayesian Q-learning

Suppose that, for every state $s$, there is a set of actions $\mathcal{A}(s)$ that can be chosen in that state. Let $Q(s, a)$ denote the expected utility of choosing action $a \in \mathcal{A}(s)$ in ...
11
votes
1answer
735 views

In Random Forest, why is a random subset of features chosen at the node level rather than at the tree level?

My Question: Why does random forest consider random subsets of features for splitting at the node level within each tree rather than at the tree level? Background: This is something of a history ...
11
votes
0answers
1k views

Rademacher complexity of logistic regression

Consider logistic regression. We have the logistic loss function, $\phi: R\rightarrow [0,1], \phi(u)=\log(1+\exp(-u))$, which is Lipschitz, and we have the linear function class $F=\{f_w:R^d \...
11
votes
1answer
520 views

Expectation Maximization Clarification

I found very helpful tutorial regarding EM algorithm. The example and the picture from the tutorial is simply brilliant. Related question about calculating probabilities how does expectation ...
10
votes
2answers
8k views

Interpreting negative cosine similarity

My question may be a silly one. So I shall apologize in advance. I was trying to use the GLOVE model pre-trained by Stanford NLP group (link). However, I noticed that my similarity results showed ...
9
votes
0answers
3k views

Backpropagation on a convolutional layer

Online tutorials describe in depth the convolution of an image with a filter, etc; However, I have not seen one that describes the backpropagation on the filter (at least visually). First let me try ...
9
votes
0answers
2k views

How can I get feature importance for Gaussian Naive Bayes classifier?

I have a dataset consisting of 4 classes and around 200 features. I have implemented a Gaussian Naive Bayes classifier. I want now calculate the importance of each feature for each pair of classes ...
9
votes
3answers
900 views

How to compare the accuracy of two different models using statistical significance

I am working on time series prediction. I have two data sets $D1=\{x_1, x_2,....x_n\}$ and $D2=\{x_n+1, x_n+2, x_n+3,...., x_n+k\}$. I have three prediction models: $M1, M2, M3$. All of those model ...
8
votes
0answers
379 views

Can Frank Harrell's method be used to obtain optimism-corrected regression coefficients?

I used a regularized (LASSO) Cox regression to estimate relapse times of patients and used Frank Harrell's bootstrapping method to obtain an optimism-corrected performance estimate of my model. ...
8
votes
2answers
1k views

Scaling the backward variable in HMM Baum-Welch

I am just trying to implement the scaled Baum-Welch algorithm and I have run into a problem where my backward variables, after scaling, are over the value of 1. Is this normal? After all, ...
8
votes
0answers
1k views

Post processing random forests using regularised regression: what about bias?

I have been playing around with post processing the results of the random forest for regression machine learning algorithm in order to try and do better than the default mean of all trees prediction. ...
7
votes
1answer
75 views

inferring most important features

Given a set of $n$ instances. For each instance I have a feature vector consisting of $m$ (numerical) features ($x_1$, $x_2$,...,$x_m$), n>>m. Moreover, for each instance I have a numerical score $y$ (...
7
votes
1answer
136 views

Why does entropy as error measure leads to overfitting?

This post on KDnuggets quoting the post by John Langford states that entropy and mutual information when used as error measures can lead to overfitting. Could you go into more details on this?
7
votes
0answers
698 views

Applying machine learning techniques to panel data

I have a panel data in which I observe 1500 companies and many individuals work for those companies for multiple periods. I have explanatory variables at both individual (e.g. race, age, education) ...
7
votes
0answers
305 views

Reinforcement *Model* Learning

Classical reinforcement learning (Q- or Sarsa-Learning) can be extended with models of the environment. These models are usually transition tables that contain the probability of arriving at a ...
7
votes
0answers
581 views

Machine learning with ordered labels

The usual method for adapting binary classifiers like various SVMs to multilabel data is one-vs-all, which assumes that labels are independent and in case of a prediction error we don't care what ...
7
votes
1answer
3k views

Unscented Kalman filter-negative covariance matrix

I have recently started working on the unscented Kalman filter. I coded the numerically stable version (i.e., square root Kalman filter) and used MATLAB for implementing. In the final update step, ...
6
votes
0answers
110 views

Understanding equation used by Hastie et al

I am trying to recreate FIGURE 3.6 from Elements of Statistical Learning. The only information about the figure is included in the caption. I am not clear on what the equation on the Y-axis means ...
6
votes
0answers
63 views

Automated ML vs the entire replicability/reproducibility crisis

There is a trend in machine learning implementations to make things easier and easier for implementers, a very natural engineering concern. Easy APIs to create any kind of model you want, easy ...
6
votes
0answers
495 views

How to pick length-scale bounds for RBC kernels in Gaussian Process Regression?

I am trying to fit GP regression models to several thousand $x, y$ pairs independantly. I am using Python's sklearn implementation with a constant kernel plus an RBF kernel plus a white noise kernel. ...
6
votes
0answers
483 views

Compatible Function Approximation Theorem in Reinforcement Learning

In the Compatible Function Approximation Theorem, the following condition is required to make the policy gradient to be exact $\nabla J(\theta) = \mathbb{E}_{\pi_{\theta}}\left [\nabla_{\theta}log\pi_{...
6
votes
1answer
620 views

Multi-label classification: Predict product category

I want to predict to which product category a product belongs. A total of 400k products need to be translated from the old (less refined) to the new product category tree. (E.g. alarm clock used to ...
6
votes
0answers
1k views

Dealing with auxiliary random variables for Mean-Field Variational Inference in Bayesian Poisson factorization

I am studying as a part of a class assignment a recent paper on Poisson factorization. Some points of the paper regarding the usage of some auxiliary variables are not clear to me. I would like to ...
6
votes
1answer
1k views

How to propagate uncertainty into the prediction of a neural network?

I have inputs $x_1\ldots x_n$ that have known $1\sigma$ uncertainties $\epsilon_1 \ldots \epsilon_n$. I am using them to predict outputs $y_1 \ldots y_m$ on a trained neural network. How can I obtain ...
6
votes
0answers
335 views

Inter-rater agreement of a gold standard dataset - a ceiling for reliable evaluation of algorithms?

In my field, a dated gold standard dataset is used to track progress in algorithm development. Now when the state-of-the-art algorithms obtain higher correlation than is the inter-rater agreement of ...
6
votes
2answers
4k views

Is it ok to get negative Cosine Similarity using LSA?

I am getting negative cosine similarity value between two documents in Latent Semantic analysis. How should it be treated?
6
votes
0answers
773 views

Classification of multiple time series and case level attributes

I'm pretty new to machine learning so wondering whether someone can help check my thinking or point me in the right direction! I need to create a classifier which can predict an outcome for a person ...
6
votes
0answers
2k views

Maximum entropy classifier and sentiment analysis

I am doing a project work in sentiment analysis (on Twitter data) using machine learning approach. In order to find the 'best' way to this I have experimented with naive Bayesian and maximum entropy ...
5
votes
1answer
5k views

What does “def” above an equals sign mean?

I am reading this: https://www.cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf and on equation (17), there is a def on top of the equal sign. What does this mean?
5
votes
0answers
57 views

Asymmetric or unequal misclassification costs in random forest

I have a general question about asymmetric costs. In machine learning problems, there are times when the cost of a false positive is different from the cost of a false negative. Accordingly, models ...
5
votes
0answers
174 views

Quantifying uncertainty when fitting a statistical model to partial effects/dependencies from a random forest (or other machine learning model)

Question: I estimate the partial dependence of $y$ on one predictor in a fitted random forest (RF). I want to now fit a parametric model to this partial dependence. How can I estimate my uncertainty ...
5
votes
0answers
142 views

When is there a free lunch?

The no free lunch theorem (NFL) states that Theorem (Wolpert and Macready 1997) Let $A$ be any learning algorithm for the task of binary classification with respect to the $0−1$ loss over a ...
5
votes
2answers
555 views

Which is the Recommended variants of neural network for tree like data structures?

Consider I have a training set where each data sample is a tree and each node has(including leaf node) its own feature vector. For example, single data sample will look this Now I have to classify <...
5
votes
0answers
5k views

Change image input size of a pre-trained convnet

maybe this question will sound a bit as a newbie one but I'd like to have some clarification. I'm using a VGG16-like convnet, pre-trained with VGG16 weights and edited top layers to work with my ...
5
votes
0answers
2k views

Deriving linear regression gradient with MSE

So I've been tinkering around with the backpropagation algorithm and to try to get a better understanding of how it works and my calculus is quite rusty. I've derived the gradient for linear ...
5
votes
0answers
348 views

How do bias, variance and overfitting relate to each other?

I'm quite new to Machine Learning, and after reading about the bias-variance tradeoff and overfitting/underfitting, several questions raised in my mind: If I have a model with 15% error on train set ...
5
votes
0answers
237 views

What is the utility/significance of PAC learnability and VC dimension?

I've been reading Shalev-Shwartz & Ben-David's book, "Understanding Machine Learning", which presents the PAC theory in its Part I. While the theory of PAC learnability does appear very elegant ...
5
votes
1answer
1k views

Using Rolling Forecast Origin Resampling in R for Neural Network Time Series

I am new to time series prediction and forecasting with neural networks and am having trouble with cross validation. I am fitting a multivariate time series. I have 236 monthly observations. I am ...
5
votes
0answers
467 views

Deep Learning vs Structured Learning

I am interested in the differences between using large, deep learning networks vs Probabilistic graphical models (PGMs), like Random Field models, for structured learning (e.g. on images, or labels of ...
5
votes
0answers
265 views

Is there an appropriate order to apply bagging and filter feature selection?

I'm training a (regression) learner on a $p \gg n$ problem, including bagging and filter feature selection (information gain). I'm in doubt though regarding the order of the procedures: Apply the ...
5
votes
0answers
7k views

True positive, false negative, true negative, false positive definitions for multiclass-multilabel classification?

I'm trying to apply some evaluation metrics to several clustering methods. I thought that I knew them basing on the multiclass confusion matrix, considering the rows as the actual classes and the ...
5
votes
0answers
1k views

Recurrent neural network for object tracking & position filtering?

Would a recurrent neural network be appropriate for object tracking tasks? Mainly I will have 3D feature vectors $(x, y, t)$ where $x$ and $y$ are the positions of an object in the image and $t$ is ...
5
votes
0answers
304 views

Interpreting hidden layer representations in ANNs

I'm using the fann library for writing an Artificial Neural Network in C++. I trained my network for the task of recognizing faces inside a set of 128x128 .png images, using three different algorithms:...
5
votes
0answers
394 views

Ideal statistical or machine learning technique to model highly cross-correlated data

I'm trying to build a model that can predict streamflow for an alpine (snowmelt-fed) watershed using snow albedo (roughly, the energy reflectance of the snow) data. Albedo controls the melt of the ...
5
votes
1answer
147 views

How to train a model when instead of a target we have a range where it is?

Often in machine learning we have a situation when target is numeric (real or integer). Each target comes with an associated input vector. The goal is to learn the mapping from the input vectors to ...
5
votes
0answers
354 views

What enforces features diversity in RBM?

I'm working on an implementation of a Restricted Boltzman Machine (RBM). I made some tests on the MNIST dataset trying to learn a representation of the digit 2. My inputs are binary images. My aim is ...
5
votes
0answers
148 views

Compressed sensing: Optimization in $L_1$ norm and total variation with fourier coefficients

I'm reading the article Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information (Candes, Romberg and Tao, 2004). In this article they are talking ...
5
votes
1answer
348 views

Unsupervised pre-training for Reinforcement Learning

Since the advent of many unsupervised learning methods, as a pretraining step for the main supervised task (mostly under the name of Deep Learning), it shouldn't be strange to ask, what is the current ...