Skip to main content

Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

6,469 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
17 votes
0 answers
2k views

Rademacher complexity of logistic regression

Consider logistic regression. We have the logistic loss function, $\phi: R\rightarrow [0,1], \phi(u)=\log(1+\exp(-u))$, which is Lipschitz, and we have the linear function class $F=\{f_w:R^d \...
axk's user avatar
  • 818
13 votes
2 answers
2k views

Scaling the backward variable in HMM Baum-Welch

I am just trying to implement the scaled Baum-Welch algorithm and I have run into a problem where my backward variables, after scaling, are over the value of 1. Is this normal? After all, ...
itzjustricky's user avatar
11 votes
0 answers
954 views

Why we really need the concept of "Local" Rademacher complexity?

Recently, I have been studying High-Dimensional Statistics: A Non-Asymptotic Viewpoint written by Martin J. Wainwright. In this book, the author uses a special complexity measure which is called Local ...
Wei-Cheng Lee's user avatar
10 votes
0 answers
568 views

When using L2 regularization outside of linear regression, do the same MAP estimation assumptions hold?

Some context is shared below, and my question is bolded at the end. MLE from observation noise In the linear regression setting, we learn model weights $\mathbf{w}$ to make scalar predictions $\hat{y}...
kdbanman's user avatar
  • 857
10 votes
0 answers
2k views

Difference between Shapley values and SHAP

The Paper regarding die SHAP value gives a formula for the Shapley Values in (4) and for SHAP values apparently (?) in (8) Still I dont really understand the difference between Shapley and SHAP ...
Quastiat's user avatar
  • 233
10 votes
0 answers
349 views

Reinforcement *Model* Learning

Classical reinforcement learning (Q- or Sarsa-Learning) can be extended with models of the environment. These models are usually transition tables that contain the probability of arriving at a ...
wehnsdaefflae's user avatar
10 votes
2 answers
770 views

How to train a model when instead of a target we have a range where it is?

Often in machine learning we have a situation when target is numeric (real or integer). Each target comes with an associated input vector. The goal is to learn the mapping from the input vectors to ...
Roman's user avatar
  • 612
9 votes
0 answers
673 views

How come the BART results are this good at the 2016 Atlantic causal inference competition?

The famous paper Dorie,2017 shows that BART performs dramatically well in causal inference. In my replication, MSE in BART can be 40% lower than MSE in other machine learning methods. But all machine ...
Ruiyuan Huang's user avatar
9 votes
1 answer
146 views

Is there a ML or DL tool that can learn to detect periodically occurring patterns in a one dimensional time series?

I am trying to create a tool that labels refrigerator temperature readings. A reading is taken every 5 minutes, and its label identifies whether of not it was taken while the refrigerator was ...
GreenBlue's user avatar
9 votes
1 answer
4k views

Bayesian hyperparameter optimization + cross-validation

I want to use Bayesian optimization to search a space of hyperparameters for a neural network model. My objective function for this optimization is validation set accuracy. In addition, I want to ...
MRF's user avatar
  • 111
8 votes
0 answers
73 views

Machine learning methods that maximize true positive rate

I am looking for a method that maximizes true positives only. Put another way, what I am looking for differs from usual classification in the sense that I would like the learning method to identify a ...
RUser4512's user avatar
  • 10.4k
8 votes
3 answers
925 views

Causal tree v. causal forest - when to use which for HTE?

Would someone be able to explain the considerations for using a causal tree versus a causal forest to estimate heterogeneous treatment effects? Is it that a causal forest is less prone to overfitting? ...
yogz123's user avatar
  • 233
8 votes
1 answer
748 views

How to predict routes using clustering data

I've been working on a ship route prediction algorithm such that given the past and current trajectory of a ship I am able to estimate the future one. The trajectories are represented as a sequence of ...
João Matos's user avatar
8 votes
0 answers
194 views

When is there a free lunch?

The no free lunch theorem (NFL) states that Theorem (Wolpert and Macready 1997) Let $A$ be any learning algorithm for the task of binary classification with respect to the $0−1$ loss over a ...
MachineEpsilon's user avatar
8 votes
0 answers
982 views

Compatible Function Approximation Theorem in Reinforcement Learning

In the Compatible Function Approximation Theorem, the following condition is required to make the policy gradient to be exact $\nabla J(\theta) = \mathbb{E}_{\pi_{\theta}}\left [\nabla_{\theta}log\pi_{...
Jiang Xiang's user avatar
8 votes
0 answers
14k views

True positive, false negative, true negative, false positive definitions for multiclass-multilabel classification?

I'm trying to apply some evaluation metrics to several clustering methods. I thought that I knew them basing on the multiclass confusion matrix, considering the rows as the actual classes and the ...
Emilio Genaro López's user avatar
8 votes
0 answers
4k views

Which standard deviation of the cross-validation score?

When doing cross-validation for model selection, I found there are many ways to quote the "standard deviation" for the cross-validation scores (here "score" means an evaluation metric e.g. accuracy, ...
xiaoxiao87's user avatar
8 votes
1 answer
6k views

Unscented Kalman filter-negative covariance matrix

I have recently started working on the unscented Kalman filter. I coded the numerically stable version (i.e., square root Kalman filter) and used MATLAB for implementing. In the final update step, ...
Sharad's user avatar
  • 81
8 votes
0 answers
259 views

How to learn similarity of typed/attributed graphs?

I have a question for graph machine learning gurus :). For this project I'm working on, I need to be able to learn similarity between typed graphs. By typed I mean that every vertex and every edge of ...
Stan Dawson's user avatar
8 votes
1 answer
89 views

Feature engineering for sheet music

I have a large dataset of digitized music scores that I'd like to use as input to a network. Initially, I'm looking to train networks to identify key signatures, tempo, dynamics, etc. from the raw ...
Bailey Parker's user avatar
7 votes
0 answers
2k views

In the attention mechanism, why are there separate weight matrices for the queries and keys?

To perform self attention over a collection of $n$ vectors stacked up into a matrix $X \in \mathbb{R}^{n \times d}$, we first obtain query, key, and value representations of these vectors via three ...
tddevlin's user avatar
  • 3,387
7 votes
0 answers
216 views

Does Fisher scoring always outperform Newton optimization?

My understanding is that Fisher scoring has several advantages over Newton raphson optimization such as Computational efficiency: if certain conditions are met (example:During MLE estimation, if link ...
Preetham_tsp's user avatar
7 votes
0 answers
2k views

Zero-inflation with sklearn and continuous target?

My current data have quite a large amount of zeros (~60%), and I'm thinking of trying to implement a zero-inflated model of sorts with sklearn. While I've used zero-inflated poisson/negative binomial ...
bjr96571's user avatar
7 votes
0 answers
723 views

Understanding Object2Vec

AWS released an interesting feature as part of the SageMaker service called Object2Vec that lets you make an embedding for search out of pretty much anything: documents, users, products, ...
Ryan Zotti's user avatar
  • 6,707
7 votes
0 answers
950 views

The extrapolation problem: model selection, performance metrics, and improvement

Machine learning models are fit to a response variable within a given range. This leads to weak and sometimes disastrous performance when it comes to instances with an actual response variable outside ...
Kinformationist's user avatar
7 votes
0 answers
165 views

How to combine noisy and noise-free datasets to train a model

Overview Suppose I have two datasets, both of which consist of rows of features and their matching labels. One of these datasets is noise-free and its labels correspond to the ground truth, but the ...
tddevlin's user avatar
  • 3,387
7 votes
0 answers
321 views

What machine learning and deep learning models are used for longitudinal studies (panel data)?

As the title suggests, I have a longitudinal database (also called panel data). (I have over 100.000 observations. The time period is X years. This means that for every year I have the values of the ...
Cedric Standaert's user avatar
7 votes
0 answers
108 views

Choosing the number of hidden layers and nodes in a Deep Belief Network

What are the recent advances and current best practices in choosing the number and size of stacked Restricted Boltzmann Machines in Deep Belief Networks ?
Robert Long's user avatar
  • 63.9k
7 votes
0 answers
8k views

Change image input size of a pre-trained convnet

maybe this question will sound a bit as a newbie one but I'd like to have some clarification. I'm using a VGG16-like convnet, pre-trained with VGG16 weights and edited top layers to work with my ...
matteodv's user avatar
  • 171
7 votes
0 answers
508 views

Machine Learning for Causal Inference with Panel Data: Possible to combine ML estimators with additive/linear terms to derive diff-in-diff estimator?

My question is motivated by the following. First consider the non-panel case, where we have two groups, the treated group ($g=t$) and the comparison group ($g=c$), and are trying to estimate an ...
Yakkanomica's user avatar
7 votes
1 answer
840 views

Multi-label classification: Predict product category

I want to predict to which product category a product belongs. A total of 400k products need to be translated from the old (less refined) to the new product category tree. (E.g. alarm clock used to ...
JobVisser21's user avatar
7 votes
0 answers
311 views

Sample Space of Machine Learning Classification "Experiment"

If you're trying to classify some input, $\mathbf{x} \in \mathbb{R}^{n}$, to one of $d$ classes using a model with parameters, $\theta$, how are you supposed to think about the experiment of learning ...
John's user avatar
  • 71
7 votes
0 answers
608 views

Deep Learning vs Structured Learning

I am interested in the differences between using large, deep learning networks vs Probabilistic graphical models (PGMs), like Random Field models, for structured learning (e.g. on images, or labels of ...
user3658307's user avatar
  • 2,224
7 votes
0 answers
2k views

Dealing with auxiliary random variables for Mean-Field Variational Inference in Bayesian Poisson factorization

I am studying as a part of a class assignment a recent paper on Poisson factorization. Some points of the paper regarding the usage of some auxiliary variables are not clear to me. I would like to ...
Alex Crychek's user avatar
7 votes
0 answers
3k views

Deriving the maximum likelihood for a generative classification model for K classes

In Christopher Bishop's book "Pattern Recognition and Machine learning", there is the following question: Consider a generative classification model for $K$ classes defined by the prior class ...
BitRiver's user avatar
  • 477
7 votes
0 answers
172 views

Compressed sensing: Optimization in $L_1$ norm and total variation with fourier coefficients

I'm reading the article Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information (Candes, Romberg and Tao, 2004). In this article they are talking ...
Roy's user avatar
  • 839
7 votes
0 answers
464 views

Any implementations of fully recurrent neural networks applied to reinforcement learning?

I've seen a single paper on the topic of adapting fully recurrent networks to a reinforcement learning setting, but according to google scholar its had no citations and no code has been released ...
zergylord's user avatar
  • 407
7 votes
0 answers
1k views

k-fold cross validation vs k times hold-out validation

I am facing the evaluation of a genetic programming algorithm. I am using the Proben1 cancer1 dataset to evaluate the models created by this algorithm. This dataset contains 699 samples, which is ...
Aktaeon's user avatar
  • 73
7 votes
0 answers
866 views

Classification of multiple time series and case level attributes

I'm pretty new to machine learning so wondering whether someone can help check my thinking or point me in the right direction! I need to create a classifier which can predict an outcome for a person ...
chrisb's user avatar
  • 925
7 votes
0 answers
187 views

Graphical nominal model

Suppose I have a set of $k$ matrices. $$ \mathbb D = A_1,A_2,...,A_k $$ Each column of $A$ is categorical vector. $$ A = v_1,v_2,...,v_n $$ I want to find the mapping $$ f: A \...
Jessica Collins's user avatar
7 votes
0 answers
2k views

Maximum entropy classifier and sentiment analysis

I am doing a project work in sentiment analysis (on Twitter data) using machine learning approach. In order to find the 'best' way to this I have experimented with naive Bayesian and maximum entropy ...
norbip's user avatar
  • 171
6 votes
0 answers
119 views

Understanding a calibration plot for lightGBM binary classifier

I wanted to assess the performance of my lightGBM classifier using a calibration plot. If I understood correctly, a calibration plot visualizes the alignment between the predicted probabilities by the ...
Programming Noob's user avatar
6 votes
1 answer
2k views

Overfitting a neural network to a single batch as a sanity check - how small a loss value is small enough and long to run for?

I'm currently developing a neural network for a regression task. Following on from the advice given in places like here, here, and here I'm attempting to overfit my model to a single batch of 5 ...
Molem7b5's user avatar
6 votes
0 answers
29 views

Is there any common characteristics for embedding?

When I was learning the embedding techniques, I tried PCA of checking the "embedding data", I found that there are no major principle components in there, i.e., variance is more "evenly ...
Haitao Du's user avatar
  • 37.2k
6 votes
0 answers
5k views

Why SVM with gamma='scale' for RBF kernel works so well?

The intuitive explanation for the gamma parameter of the RBF kernel in SVMs is the following: Intuitively, the gamma parameter ...
kuzand's user avatar
  • 266
6 votes
0 answers
256 views

Interpretation of the alpha parameter in the Rational Quadratic Kernel

I've been working on Gaussian processes and a problem that keeps bugging me is the alpha parameter on the rational quadratic kernel I know that the rational quadratic is an infinite sum of squared ...
OvertopGP's user avatar
6 votes
0 answers
2k views

Interpreting SHAP Dependence Plot for Categorical Variables

I'm reading about the use of Shapley values for explaining complex machine learning models and I'm confused about how I should interpret the SHAP independence plot in the case of a categorical ...
Blg Khalil's user avatar
6 votes
0 answers
119 views

Why is Rademacher complexity defined the way it is?

For reference, this is the definition of empirical Rademacher complexity from Foundations of Machine Learning (page 30): Let $\mathcal{G}$ be a family of functions mapping from $\mathcal{Z}$ to $[...
Milos's user avatar
  • 1,099
6 votes
1 answer
197 views

XGBOOST objective function derivation algebra

I need some help please with the derivation of xgboost objective function. I am following this online tutorial (Math behind GBM and XGBoost) How do you go from here $$ loss = \sum_{i=1}^{n} \left( ...
Edv Beq's user avatar
  • 713
6 votes
0 answers
2k views

Why do we need the gamma parameter in the polynomial kernel of SVMs?

The polynomial kernel is sometimes defined as just: $$ K(x,y):=(\left<x,y\right>+c)^d $$ with two parameters: the degree $d$ and constant coefficient $c$. But others (e.g., libsvm, and sklearn ...
Erich Schubert's user avatar

1
2 3 4 5
130