Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

6,227 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
28 votes
0 answers
2k views

How does a Relevance Vector Machine (RVM) work?

Relevance Vector Machines (RVMs) are really interesting models when contrasted with the highly geometrical (and popular) SVMs. In the light of a question like How does a Support Vector Machine (SVM) ...
user avatar
  • 15.7k
14 votes
0 answers
2k views

Rademacher complexity of logistic regression

Consider logistic regression. We have the logistic loss function, $\phi: R\rightarrow [0,1], \phi(u)=\log(1+\exp(-u))$, which is Lipschitz, and we have the linear function class $F=\{f_w:R^d \...
user avatar
  • 736
12 votes
0 answers
737 views

what is the mistake of convergence proof in Adam

Sashank J. Reddi et. al in their paper "On the convergence of Adam and beyond" say that, Adam's proof of convergence as stated in original paper is wrong. More than that, they point out that the value ...
user avatar
12 votes
2 answers
2k views

Scaling the backward variable in HMM Baum-Welch

I am just trying to implement the scaled Baum-Welch algorithm and I have run into a problem where my backward variables, after scaling, are over the value of 1. Is this normal? After all, ...
user avatar
11 votes
0 answers
10k views

Deriving linear regression gradient with MSE

So I've been tinkering around with the backpropagation algorithm and to try to get a better understanding of how it works and my calculus is quite rusty. I've derived the gradient for linear ...
user avatar
  • 273
10 votes
2 answers
2k views

Applying machine learning techniques to panel data

I have a panel data in which I observe 1500 companies and many individuals work for those companies for multiple periods. I have explanatory variables at both individual (e.g. race, age, education) ...
user avatar
  • 273
10 votes
0 answers
339 views

Reinforcement *Model* Learning

Classical reinforcement learning (Q- or Sarsa-Learning) can be extended with models of the environment. These models are usually transition tables that contain the probability of arriving at a ...
user avatar
9 votes
1 answer
11k views

What does "def" above an equals sign mean?

I am reading this: https://www.cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf and on equation (17), there is a def on top of the equal sign. What does this mean?
user avatar
9 votes
1 answer
293 views

Mini-Batch Gradient Descent - Why does sampling with replacement work?

When sampling the data, either one at a time (as in online learning), or in mini-batches, there exist gradient descent methods which sample with replacement and without replacement. For Mini-Batch ...
user avatar
9 votes
1 answer
117 views

Is there a ML or DL tool that can learn to detect periodically occurring patterns in a one dimensional time series?

I am trying to create a tool that labels refrigerator temperature readings. A reading is taken every 5 minutes, and its label identifies whether of not it was taken while the refrigerator was ...
user avatar
9 votes
1 answer
478 views

How to train a model when instead of a target we have a range where it is?

Often in machine learning we have a situation when target is numeric (real or integer). Each target comes with an associated input vector. The goal is to learn the mapping from the input vectors to ...
user avatar
  • 973
8 votes
2 answers
808 views

Why are the embeddings of tokens multiplied by $\sqrt D$ (note not divided by square root of D) in a transformer?

Why does the transformer tutorial in PyTorch have a multiplication by sqrt number of inputs? I know there is a division by sqrt(D) in the multiheaded self attention, but why is there something similar ...
user avatar
8 votes
0 answers
213 views

Why is BART so accurate in causal inference?

The famous paper Dorie,2017 shows that BART performs dramatically well in causal inference. In my replication, MSE in BART can be 40% lower than MSE in other machine learning methods. But all machine ...
user avatar
8 votes
2 answers
350 views

Feature Engineering : combine a categorical Feature and a continuous Feature

When we analyze data , we can observe several variables that may contain mutual information. For an example , There can be a binary variable such as Y=Have you ever smoke ? And then there will be a ...
user avatar
8 votes
0 answers
371 views

When using L2 regularization outside of linear regression, do the same MAP estimation assumptions hold?

Some context is shared below, and my question is bolded at the end. In the linear regression setting, we learn model weights $\hat{\mathbf{w}}$ to make predictions $\mathbf{\hat{y}}$ from new samples ...
user avatar
  • 560
8 votes
0 answers
834 views

Difference between Shapley values and SHAP

The Paper regarding die SHAP value gives a formula for the Shapley Values in (4) and for SHAP values apparently (?) in (8) Still I dont really understand the difference between Shapley and SHAP ...
user avatar
  • 203
8 votes
0 answers
177 views

When is there a free lunch?

The no free lunch theorem (NFL) states that Theorem (Wolpert and Macready 1997) Let $A$ be any learning algorithm for the task of binary classification with respect to the $0−1$ loss over a ...
user avatar
8 votes
0 answers
892 views

Compatible Function Approximation Theorem in Reinforcement Learning

In the Compatible Function Approximation Theorem, the following condition is required to make the policy gradient to be exact $\nabla J(\theta) = \mathbb{E}_{\pi_{\theta}}\left [\nabla_{\theta}log\pi_{...
user avatar
8 votes
0 answers
3k views

Which standard deviation of the cross-validation score?

When doing cross-validation for model selection, I found there are many ways to quote the "standard deviation" for the cross-validation scores (here "score" means an evaluation metric e.g. accuracy, ...
user avatar
8 votes
2 answers
6k views

Is it ok to get negative Cosine Similarity using LSA?

I am getting negative cosine similarity value between two documents in Latent Semantic analysis. How should it be treated?
user avatar
8 votes
0 answers
776 views

Machine learning with ordered labels

The usual method for adapting binary classifiers like various SVMs to multilabel data is one-vs-all, which assumes that labels are independent and in case of a prediction error we don't care what ...
user avatar
8 votes
1 answer
5k views

Unscented Kalman filter-negative covariance matrix

I have recently started working on the unscented Kalman filter. I coded the numerically stable version (i.e., square root Kalman filter) and used MATLAB for implementing. In the final update step, ...
user avatar
  • 81
7 votes
0 answers
131 views

How to combine noisy and noise-free datasets to train a model

Overview Suppose I have two datasets, both of which consist of rows of features and their matching labels. One of these datasets is noise-free and its labels correspond to the ground truth, but the ...
user avatar
  • 3,205
7 votes
1 answer
491 views

How to predict routes using clustering data

I've been working on a ship route prediction algorithm such that given the past and current trajectory of a ship I am able to estimate the future one. The trajectories are represented as a sequence of ...
user avatar
7 votes
0 answers
7k views

Change image input size of a pre-trained convnet

maybe this question will sound a bit as a newbie one but I'd like to have some clarification. I'm using a VGG16-like convnet, pre-trained with VGG16 weights and edited top layers to work with my ...
user avatar
  • 171
7 votes
3 answers
676 views

How do bias, variance and overfitting relate to each other?

I'm quite new to Machine Learning, and after reading about the bias-variance tradeoff and overfitting/underfitting, several questions raised in my mind: If I have a model with 15% error on train set ...
user avatar
  • 101
7 votes
0 answers
426 views

Machine Learning for Causal Inference with Panel Data: Possible to combine ML estimators with additive/linear terms to derive diff-in-diff estimator?

My question is motivated by the following. First consider the non-panel case, where we have two groups, the treated group ($g=t$) and the comparison group ($g=c$), and are trying to estimate an ...
user avatar
7 votes
1 answer
753 views

Multi-label classification: Predict product category

I want to predict to which product category a product belongs. A total of 400k products need to be translated from the old (less refined) to the new product category tree. (E.g. alarm clock used to ...
user avatar
7 votes
1 answer
561 views

Deep Learning vs Structured Learning

I am interested in the differences between using large, deep learning networks vs Probabilistic graphical models (PGMs), like Random Field models, for structured learning (e.g. on images, or labels of ...
user avatar
  • 1,804
7 votes
0 answers
13k views

True positive, false negative, true negative, false positive definitions for multiclass-multilabel classification?

I'm trying to apply some evaluation metrics to several clustering methods. I thought that I knew them basing on the multiclass confusion matrix, considering the rows as the actual classes and the ...
user avatar
7 votes
0 answers
2k views

Dealing with auxiliary random variables for Mean-Field Variational Inference in Bayesian Poisson factorization

I am studying as a part of a class assignment a recent paper on Poisson factorization. Some points of the paper regarding the usage of some auxiliary variables are not clear to me. I would like to ...
user avatar
7 votes
0 answers
3k views

Deriving the maximum likelihood for a generative classification model for K classes

In Christopher Bishop's book "Pattern Recognition and Machine learning", there is the following question: Consider a generative classification model for $K$ classes defined by the prior class ...
user avatar
  • 467
7 votes
0 answers
160 views

Compressed sensing: Optimization in $L_1$ norm and total variation with fourier coefficients

I'm reading the article Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information (Candes, Romberg and Tao, 2004). In this article they are talking ...
user avatar
  • 729
7 votes
0 answers
446 views

Any implementations of fully recurrent neural networks applied to reinforcement learning?

I've seen a single paper on the topic of adapting fully recurrent networks to a reinforcement learning setting, but according to google scholar its had no citations and no code has been released ...
user avatar
  • 407
7 votes
0 answers
1k views

k-fold cross validation vs k times hold-out validation

I am facing the evaluation of a genetic programming algorithm. I am using the Proben1 cancer1 dataset to evaluate the models created by this algorithm. This dataset contains 699 samples, which is ...
user avatar
  • 73
7 votes
0 answers
213 views

How to learn similarity of typed/attributed graphs?

I have a question for graph machine learning gurus :). For this project I'm working on, I need to be able to learn similarity between typed graphs. By typed I mean that every vertex and every edge of ...
user avatar
7 votes
0 answers
837 views

Classification of multiple time series and case level attributes

I'm pretty new to machine learning so wondering whether someone can help check my thinking or point me in the right direction! I need to create a classifier which can predict an outcome for a person ...
user avatar
  • 835
7 votes
0 answers
161 views

Graphical nominal model

Suppose I have a set of $k$ matrices. $$ \mathbb D = A_1,A_2,...,A_k $$ Each column of $A$ is categorical vector. $$ A = v_1,v_2,...,v_n $$ I want to find the mapping $$ f: A \...
user avatar
7 votes
0 answers
2k views

Maximum entropy classifier and sentiment analysis

I am doing a project work in sentiment analysis (on Twitter data) using machine learning approach. In order to find the 'best' way to this I have experimented with naive Bayesian and maximum entropy ...
user avatar
  • 171
6 votes
0 answers
100 views

Why convert spectrogram to RGB for machine learning?

I've seen a few publications that feed an RGB image of a spectrogram to a neural net, and someone claiming a network does better with RGB than grayscale. A spectrogram is fundamentally a 2D ...
user avatar
6 votes
0 answers
107 views

Lebesgue-Stieltjes integration by parts on a half-open interval

I have run into a problem in a proof of the bound for the rate convergence of an empirical risk function based on unbounded loss to the true model risk (Vapnik, Statistical Learning Theory, Theorem 5....
user avatar
  • 191
6 votes
0 answers
105 views

What does non-linearity mean when using only binary predictors?

I'm working on a project that uses genetic data. The dataset has thousands of predictors that are all binary (does the person have a certain letter in a certain position in their genetic code: yes/no)....
user avatar
6 votes
0 answers
1k views

Interpreting SHAP Dependence Plot for Categorical Variables

I'm reading about the use of Shapley values for explaining complex machine learning models and I'm confused about how I should interpret the SHAP independence plot in the case of a categorical ...
user avatar
6 votes
0 answers
115 views

Does Fisher scoring always outperform Newton optimization?

My understanding is that Fisher scoring has several advantages over Newton raphson optimization such as Computational efficiency: if certain conditions are met (example:During MLE estimation, if link ...
user avatar
6 votes
0 answers
559 views

Understanding Object2Vec

AWS released an interesting feature as part of the SageMaker service called Object2Vec that lets you make an embedding for search out of pretty much anything: documents, users, products, ...
user avatar
  • 5,997
6 votes
1 answer
161 views

XGBOOST objective function derivation algebra

I need some help please with the derivation of xgboost objective function. I am following this online tutorial (Math behind GBM and XGBoost) How do you go from here $$ loss = \sum_{i=1}^{n} \left( ...
user avatar
  • 721
6 votes
0 answers
531 views

The extrapolation problem: model selection, performance metrics, and improvement

Machine learning models are fit to a response variable within a given range. This leads to weak and sometimes disastrous performance when it comes to instances with an actual response variable outside ...
user avatar
6 votes
0 answers
493 views

Why we really need the concept of "Local" Rademacher complexity?

Recently, I have been studying High-Dimensional Statistics: A Non-Asymptotic Viewpoint written by Martin J. Wainwright. In this book, the author uses a special complexity measure which is called Local ...
user avatar
6 votes
0 answers
54 views

Feature engineering for sheet music

I have a large dataset of digitized music scores that I'd like to use as input to a network. Initially, I'm looking to train networks to identify key signatures, tempo, dynamics, etc. from the raw ...
user avatar
6 votes
0 answers
287 views

What machine learning and deep learning models are used for longitudinal studies (panel data)?

As the title suggests, I have a longitudinal database (also called panel data). (I have over 100.000 observations. The time period is X years. This means that for every year I have the values of the ...
user avatar

1
2 3 4 5
125