Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

5,314 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
33
votes
1answer
1k views

Link Anomaly Detection in Temporal Network

I came across this paper that uses link anomaly detection to predict trending topics, and I found it incredibly intriguing: The paper is "Discovering Emerging Topics in Social Streams via Link Anomaly ...
23
votes
0answers
1k views

How does a Relevance Vector Machine (RVM) work?

Relevance Vector Machines (RVMs) are really interesting models when contrasted with the highly geometrical (and popular) SVMs. In the light of a question like How does a Support Vector Machine (SVM) ...
19
votes
1answer
5k views

Backpropagation on a convolutional layer

Online tutorials describe in depth the convolution of an image with a filter, etc; However, I have not seen one that describes the backpropagation on the filter (at least visually). First let me try ...
13
votes
0answers
931 views

Bayesian Q-learning

Suppose that, for every state $s$, there is a set of actions $\mathcal{A}(s)$ that can be chosen in that state. Let $Q(s, a)$ denote the expected utility of choosing action $a \in \mathcal{A}(s)$ in ...
12
votes
1answer
324 views

Why does entropy as error measure leads to overfitting?

This post on KDnuggets quoting the post by John Langford states that entropy and mutual information when used as error measures can lead to overfitting. Could you go into more details on this?
11
votes
0answers
1k views

Rademacher complexity of logistic regression

Consider logistic regression. We have the logistic loss function, $\phi: R\rightarrow [0,1], \phi(u)=\log(1+\exp(-u))$, which is Lipschitz, and we have the linear function class $F=\{f_w:R^d \...
10
votes
0answers
755 views

How to pick length-scale bounds for RBC kernels in Gaussian Process Regression?

I am trying to fit GP regression models to several thousand $x, y$ pairs independantly. I am using Python's sklearn implementation with a constant kernel plus an RBF kernel plus a white noise kernel. ...
10
votes
0answers
3k views

How can I get feature importance for Gaussian Naive Bayes classifier?

I have a dataset consisting of 4 classes and around 200 features. I have implemented a Gaussian Naive Bayes classifier. I want now calculate the importance of each feature for each pair of classes ...
10
votes
2answers
2k views

Scaling the backward variable in HMM Baum-Welch

I am just trying to implement the scaled Baum-Welch algorithm and I have run into a problem where my backward variables, after scaling, are over the value of 1. Is this normal? After all, ...
9
votes
0answers
432 views

Can Frank Harrell's method be used to obtain optimism-corrected regression coefficients?

I used a regularized (LASSO) Cox regression to estimate relapse times of patients and used Frank Harrell's bootstrapping method to obtain an optimism-corrected performance estimate of my model. ...
8
votes
0answers
139 views

Understanding equation used by Hastie et al

I am trying to recreate FIGURE 3.6 from Elements of Statistical Learning. The only information about the figure is included in the caption. I am not clear on what the equation on the Y-axis means ...
8
votes
1answer
920 views

Applying machine learning techniques to panel data

I have a panel data in which I observe 1500 companies and many individuals work for those companies for multiple periods. I have explanatory variables at both individual (e.g. race, age, education) ...
8
votes
0answers
1k views

Post processing random forests using regularised regression: what about bias?

I have been playing around with post processing the results of the random forest for regression machine learning algorithm in order to try and do better than the default mean of all trees prediction. ...
7
votes
1answer
7k views

What does “def” above an equals sign mean?

I am reading this: https://www.cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf and on equation (17), there is a def on top of the equal sign. What does this mean?
7
votes
0answers
79 views

Is there a ML or DL tool that can learn to detect periodically occurring patterns in a one dimensional time series?

I am trying to create a tool that labels refrigerator temperature readings. A reading is taken every 5 minutes, and its label identifies whether of not it was taken while the refrigerator was ...
7
votes
1answer
90 views

inferring most important features

Given a set of $n$ instances. For each instance I have a feature vector consisting of $m$ (numerical) features ($x_1$, $x_2$,...,$x_m$), n>>m. Moreover, for each instance I have a numerical score $y$ (...
7
votes
0answers
311 views

Reinforcement *Model* Learning

Classical reinforcement learning (Q- or Sarsa-Learning) can be extended with models of the environment. These models are usually transition tables that contain the probability of arriving at a ...
7
votes
1answer
505 views

Deep Learning vs Structured Learning

I am interested in the differences between using large, deep learning networks vs Probabilistic graphical models (PGMs), like Random Field models, for structured learning (e.g. on images, or labels of ...
7
votes
0answers
658 views

Machine learning with ordered labels

The usual method for adapting binary classifiers like various SVMs to multilabel data is one-vs-all, which assumes that labels are independent and in case of a prediction error we don't care what ...
7
votes
1answer
4k views

Unscented Kalman filter-negative covariance matrix

I have recently started working on the unscented Kalman filter. I coded the numerically stable version (i.e., square root Kalman filter) and used MATLAB for implementing. In the final update step, ...
6
votes
1answer
85 views

Feature Engineering : combine a categorical Feature and a continuous Feature

When we analyze data , we can observe several variables that may contain mutual information. For an example , There can be a binary variable such as Y=Have you ever smoke ? And then there will be a ...
6
votes
0answers
101 views

Moments of $Y=X_1 + X_2 X_3 + X_4 X_5 X_6 +\cdots$

The $X_i$'s are i.i.d. and $X$ denotes any of these random variables. We assume here that $|E(X)|<1$ to guarantee convergence. I am interested in particular in the third moment $E(Y^3)$. For the ...
6
votes
0answers
51 views

How to combine noisy and noise-free datasets to train a model

Overview Suppose I have two datasets, both of which consist of rows of features and their matching labels. One of these datasets is noise-free and its labels correspond to the ground truth, but the ...
6
votes
0answers
7k views

Change image input size of a pre-trained convnet

maybe this question will sound a bit as a newbie one but I'd like to have some clarification. I'm using a VGG16-like convnet, pre-trained with VGG16 weights and edited top layers to work with my ...
6
votes
0answers
5k views

Deriving linear regression gradient with MSE

So I've been tinkering around with the backpropagation algorithm and to try to get a better understanding of how it works and my calculus is quite rusty. I've derived the gradient for linear ...
6
votes
0answers
3k views

How does XGBoost/lightGBM evaluate ndcg metric for ranking

I am currently running tests between XGBoost/lightGBM for their ability to rank items. I am reproducing the benchmarks presented here: https://github.com/guolinke/boosting_tree_benchmarks. I have ...
6
votes
1answer
428 views

How do bias, variance and overfitting relate to each other?

I'm quite new to Machine Learning, and after reading about the bias-variance tradeoff and overfitting/underfitting, several questions raised in my mind: If I have a model with 15% error on train set ...
6
votes
0answers
697 views

Compatible Function Approximation Theorem in Reinforcement Learning

In the Compatible Function Approximation Theorem, the following condition is required to make the policy gradient to be exact $\nabla J(\theta) = \mathbb{E}_{\pi_{\theta}}\left [\nabla_{\theta}log\pi_{...
6
votes
1answer
676 views

Multi-label classification: Predict product category

I want to predict to which product category a product belongs. A total of 400k products need to be translated from the old (less refined) to the new product category tree. (E.g. alarm clock used to ...
6
votes
0answers
4k views

Bagging of xgboost

The extreme-gradient boosting algorithm seems to be widely applied these days. I often have the feeling that boosted models tend to overfit. I know that there are parameters in the algorithm to ...
6
votes
0answers
1k views

Dealing with auxiliary random variables for Mean-Field Variational Inference in Bayesian Poisson factorization

I am studying as a part of a class assignment a recent paper on Poisson factorization. Some points of the paper regarding the usage of some auxiliary variables are not clear to me. I would like to ...
6
votes
0answers
400 views

Inter-rater agreement of a gold standard dataset - a ceiling for reliable evaluation of algorithms?

In my field, a dated gold standard dataset is used to track progress in algorithm development. Now when the state-of-the-art algorithms obtain higher correlation than is the inter-rater agreement of ...
6
votes
2answers
5k views

Is it ok to get negative Cosine Similarity using LSA?

I am getting negative cosine similarity value between two documents in Latent Semantic analysis. How should it be treated?
6
votes
1answer
201 views

How to train a model when instead of a target we have a range where it is?

Often in machine learning we have a situation when target is numeric (real or integer). Each target comes with an associated input vector. The goal is to learn the mapping from the input vectors to ...
6
votes
0answers
1k views

k-fold cross validation vs k times hold-out validation

I am facing the evaluation of a genetic programming algorithm. I am using the Proben1 cancer1 dataset to evaluate the models created by this algorithm. This dataset contains 699 samples, which is ...
6
votes
0answers
794 views

Classification of multiple time series and case level attributes

I'm pretty new to machine learning so wondering whether someone can help check my thinking or point me in the right direction! I need to create a classifier which can predict an outcome for a person ...
6
votes
0answers
2k views

Maximum entropy classifier and sentiment analysis

I am doing a project work in sentiment analysis (on Twitter data) using machine learning approach. In order to find the 'best' way to this I have experimented with naive Bayesian and maximum entropy ...
5
votes
1answer
160 views

Time Series Regressor Selection

I am interest in a (multivariate) algorithm to identify relevant regressors (which are itself time series) to forecast a time series of interest. The question is worded in general terms because this ...
5
votes
0answers
40 views

Feature engineering for sheet music

I have a large dataset of digitized music scores that I'd like to use as input to a network. Initially, I'm looking to train networks to identify key signatures, tempo, dynamics, etc. from the raw ...
5
votes
0answers
357 views

“Hierarchical” Random forests?

Background I am using Random Forest to classify ~900 objects based on a large number (> 80) predictors. I split these 70:30 for training and testing. The overall model does fairly well, giving an ...
5
votes
1answer
92 views

Conterfactual estimation in machine learning model

There are various techniques to build counterfactual estimations of certain variables for linear models in observational studies. Some of those are based on comparing the change in the predicted ...
5
votes
0answers
59 views

Choosing the number of hidden layers and nodes in a Deep Belief Network

What are the recent advances and current best practices in choosing the number and size of stacked Restricted Boltzmann Machines in Deep Belief Networks ?
5
votes
0answers
146 views

A few questions regarding the practice of heterogeneous treatment effect analysis (a.k.a, interaction detection or subgroup analysis) methods

Imagine I am looking at a randomized experiment between a control and one or more treatment conditions. For example, I have a treatment that aims to get people out of debt. I randomize people to ...
5
votes
0answers
208 views

Quantifying uncertainty when fitting a statistical model to partial effects/dependencies from a random forest (or other machine learning model)

Question: I estimate the partial dependence of $y$ on one predictor in a fitted random forest (RF). I want to now fit a parametric model to this partial dependence. How can I estimate my uncertainty ...
5
votes
0answers
649 views

Why no one talks about stochastic conjugate gradient descent?

As is known to all, stochastic gradient descent is a popular optimizer in machine learning. There have been many variants of SGD. However, it has come to my attention that no one talks about the ...
5
votes
0answers
155 views

When is there a free lunch?

The no free lunch theorem (NFL) states that Theorem (Wolpert and Macready 1997) Let $A$ be any learning algorithm for the task of binary classification with respect to the $0−1$ loss over a ...
5
votes
1answer
439 views

Measuring the bias-variance tradeoff

Does anyone know of a metric that quantifies the bias-variance tradeoff of a given fitted model? I'm not talking about measuring the MSE in cross validation, I'm interested in a single generic or ...
5
votes
1answer
396 views

What is the utility/significance of PAC learnability and VC dimension?

I've been reading Shalev-Shwartz & Ben-David's book, "Understanding Machine Learning", which presents the PAC theory in its Part I. While the theory of PAC learnability does appear very elegant ...
5
votes
1answer
2k views

Using Rolling Forecast Origin Resampling in R for Neural Network Time Series

I am new to time series prediction and forecasting with neural networks and am having trouble with cross validation. I am fitting a multivariate time series. I have 236 monthly observations. I am ...
5
votes
0answers
263 views

Sample Space of Machine Learning Classification “Experiment”

If you're trying to classify some input, $\mathbf{x} \in \mathbb{R}^{n}$, to one of $d$ classes using a model with parameters, $\theta$, how are you supposed to think about the experiment of learning ...

1
2 3 4 5
107