Questions tagged [machine-learning]
Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.
6,556
questions with no upvoted or accepted answers
16
votes
0
answers
1k
views
what is the mistake of convergence proof in Adam
Sashank J. Reddi et. al in their paper "On the convergence of Adam and beyond" say that, Adam's proof of convergence as stated in original paper is wrong. More than that, they point out that the value ...
- 303
16
votes
0
answers
2k
views
Rademacher complexity of logistic regression
Consider logistic regression. We have the logistic loss function,
$\phi: R\rightarrow [0,1], \phi(u)=\log(1+\exp(-u))$, which is Lipschitz, and we have the linear function class $F=\{f_w:R^d \...
- 766
12
votes
2
answers
2k
views
Scaling the backward variable in HMM Baum-Welch
I am just trying to implement the scaled Baum-Welch algorithm and I have run into a problem where my backward variables, after scaling, are over the value of 1. Is this normal? After all, ...
- 121
10
votes
0
answers
484
views
When using L2 regularization outside of linear regression, do the same MAP estimation assumptions hold?
Some context is shared below, and my question is bolded at the end.
MLE from observation noise
In the linear regression setting, we learn model weights $\mathbf{w}$ to make scalar predictions $\hat{y}...
- 777
10
votes
0
answers
346
views
Reinforcement *Model* Learning
Classical reinforcement learning (Q- or Sarsa-Learning) can be extended with models of the environment. These models are usually transition tables that contain the probability of arriving at a ...
- 536
10
votes
2
answers
661
views
How to train a model when instead of a target we have a range where it is?
Often in machine learning we have a situation when target is numeric (real or integer). Each target comes with an associated input vector. The goal is to learn the mapping from the input vectors to ...
- 485
9
votes
1
answer
14k
views
What does "def" above an equals sign mean?
I am reading this:
https://www.cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf
and on equation (17), there is a def on top of the equal sign. What does this mean?
- 91
9
votes
0
answers
486
views
Why is BART so accurate in causal inference?
The famous paper Dorie,2017 shows that BART performs dramatically well in causal inference. In my replication, MSE in BART can be 40% lower than MSE in other machine learning methods.
But all machine ...
- 553
9
votes
1
answer
135
views
Is there a ML or DL tool that can learn to detect periodically occurring patterns in a one dimensional time series?
I am trying to create a tool that labels refrigerator temperature readings. A reading is taken every 5 minutes, and its label identifies whether of not it was taken while the refrigerator was ...
- 91
9
votes
0
answers
1k
views
Difference between Shapley values and SHAP
The Paper regarding die SHAP value gives a formula for the Shapley Values in (4) and for SHAP values apparently (?) in (8)
Still I dont really understand the difference between Shapley and SHAP ...
- 213
8
votes
2
answers
579
views
Feature Engineering : combine a categorical Feature and a continuous Feature
When we analyze data , we can observe several variables that may contain mutual information. For an example , There can be a binary variable such as Y=Have you ever smoke ? And then there will be a ...
- 859
8
votes
0
answers
189
views
When is there a free lunch?
The no free lunch theorem (NFL) states that
Theorem (Wolpert and Macready 1997) Let $A$ be any learning algorithm for the task of binary classification with respect to the
$0−1$ loss over a ...
- 2,806
8
votes
0
answers
939
views
Compatible Function Approximation Theorem in Reinforcement Learning
In the Compatible Function Approximation Theorem, the following condition is required to make the policy gradient to be exact $\nabla J(\theta) = \mathbb{E}_{\pi_{\theta}}\left [\nabla_{\theta}log\pi_{...
- 452
8
votes
0
answers
13k
views
True positive, false negative, true negative, false positive definitions for multiclass-multilabel classification?
I'm trying to apply some evaluation metrics to several clustering methods. I thought that I knew them basing on the multiclass confusion matrix, considering the rows as the actual classes and the ...
8
votes
0
answers
3k
views
Which standard deviation of the cross-validation score?
When doing cross-validation for model selection, I found there are many ways to quote the "standard deviation" for the cross-validation scores (here "score" means an evaluation metric e.g. accuracy, ...
- 421
8
votes
1
answer
5k
views
Unscented Kalman filter-negative covariance matrix
I have recently started working on the unscented Kalman filter. I coded the numerically stable version (i.e., square root Kalman filter) and used MATLAB for implementing. In the final update step, ...
- 81
8
votes
0
answers
236
views
How to learn similarity of typed/attributed graphs?
I have a question for graph machine learning gurus :). For this project I'm working on, I need to be able to learn similarity between typed graphs. By typed I mean that every vertex and every edge of ...
- 81
8
votes
1
answer
82
views
Feature engineering for sheet music
I have a large dataset of digitized music scores that I'd like to use as input to a network. Initially, I'm looking to train networks to identify key signatures, tempo, dynamics, etc. from the raw ...
- 181
7
votes
1
answer
5k
views
z-score VS min-max normalization
Working with data that use different dimensions, you do not want that one dimension dominate.
This means feature scaling!
A very intuitive way is to use min-max scaling so you scale everything between ...
- 121
7
votes
0
answers
2k
views
Zero-inflation with sklearn and continuous target?
My current data have quite a large amount of zeros (~60%), and I'm thinking of trying to implement a zero-inflated model of sorts with sklearn. While I've used zero-inflated poisson/negative binomial ...
- 71
7
votes
0
answers
150
views
How to combine noisy and noise-free datasets to train a model
Overview
Suppose I have two datasets, both of which consist of rows of features and their matching labels. One of these datasets is noise-free and its labels correspond to the ground truth, but the ...
- 3,317
7
votes
0
answers
316
views
What machine learning and deep learning models are used for longitudinal studies (panel data)?
As the title suggests, I have a longitudinal database (also called panel data). (I have over 100.000 observations. The time period is X years. This means that for every year I have the values of the ...
- 101
7
votes
0
answers
743
views
"Hierarchical" Random forests?
Background
I am using Random Forest to classify ~900 objects based on a large number (> 80) predictors. I split these 70:30 for training and testing. The overall model does fairly well, giving an ...
- 193
7
votes
1
answer
639
views
How to predict routes using clustering data
I've been working on a ship route prediction algorithm such that given the past and current trajectory of a ship I am able to estimate the future one. The trajectories are represented as a sequence of ...
- 193
7
votes
0
answers
7k
views
Change image input size of a pre-trained convnet
maybe this question will sound a bit as a newbie one but I'd like to have some clarification.
I'm using a VGG16-like convnet, pre-trained with VGG16 weights and edited top layers to work with my ...
- 171
7
votes
0
answers
479
views
Machine Learning for Causal Inference with Panel Data: Possible to combine ML estimators with additive/linear terms to derive diff-in-diff estimator?
My question is motivated by the following. First consider the non-panel case, where we have two groups, the treated group ($g=t$) and the comparison group ($g=c$), and are trying to estimate an ...
- 473
7
votes
1
answer
812
views
Multi-label classification: Predict product category
I want to predict to which product category a product belongs. A total of 400k products need to be translated from the old (less refined) to the new product category tree. (E.g. alarm clock used to ...
- 71
7
votes
1
answer
587
views
Deep Learning vs Structured Learning
I am interested in the differences between using large, deep learning networks vs Probabilistic graphical models (PGMs), like Random Field models, for structured learning (e.g. on images, or labels of ...
- 2,134
7
votes
0
answers
2k
views
Dealing with auxiliary random variables for Mean-Field Variational Inference in Bayesian Poisson factorization
I am studying as a part of a class assignment a recent paper on Poisson factorization. Some points of the paper regarding the usage of some auxiliary variables are not clear to me. I would like to ...
- 81
7
votes
0
answers
3k
views
Deriving the maximum likelihood for a generative classification model for K classes
In Christopher Bishop's book "Pattern Recognition and Machine learning", there is the following question:
Consider a generative classification model for $K$ classes defined by the prior class ...
- 467
7
votes
0
answers
167
views
Compressed sensing: Optimization in $L_1$ norm and total variation with fourier coefficients
I'm reading the article Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information (Candes, Romberg and Tao, 2004).
In this article they are talking ...
- 799
7
votes
0
answers
458
views
Any implementations of fully recurrent neural networks applied to reinforcement learning?
I've seen a single paper on the topic of adapting fully recurrent networks to a reinforcement learning setting, but according to google scholar its had no citations and no code has been released ...
- 407
7
votes
0
answers
1k
views
k-fold cross validation vs k times hold-out validation
I am facing the evaluation of a genetic programming algorithm. I am using the Proben1 cancer1 dataset to evaluate the models created by this algorithm. This dataset contains 699 samples, which is ...
- 73
7
votes
0
answers
854
views
Classification of multiple time series and case level attributes
I'm pretty new to machine learning so wondering whether someone can help check my thinking or point me in the right direction!
I need to create a classifier which can predict an outcome for a person ...
- 905
7
votes
0
answers
169
views
Graphical nominal model
Suppose I have a set of $k$ matrices.
$$
\mathbb D = A_1,A_2,...,A_k
$$
Each column of $A$ is categorical vector.
$$
A = v_1,v_2,...,v_n
$$
I want to find the mapping
$$
f: A \...
- 4,121
7
votes
0
answers
2k
views
Maximum entropy classifier and sentiment analysis
I am doing a project work in sentiment analysis (on Twitter data) using machine learning approach. In order to find the 'best' way to this I have experimented with naive Bayesian and maximum entropy ...
- 171
7
votes
1
answer
3k
views
Bayesian hyperparameter optimization + cross-validation
I want to use Bayesian optimization to search a space of hyperparameters for a neural network model. My objective function for this optimization is validation set accuracy.
In addition, I want to ...
- 91
6
votes
0
answers
197
views
Interpretation of the alpha parameter in the Rational Quadratic Kernel
I've been working on Gaussian processes and a problem that keeps bugging me is the alpha parameter on the rational quadratic kernel
I know that the rational quadratic is an infinite sum of squared ...
- 61
6
votes
0
answers
2k
views
Interpreting SHAP Dependence Plot for Categorical Variables
I'm reading about the use of Shapley values for explaining complex machine learning models and I'm confused about how I should interpret the SHAP independence plot in the case of a categorical ...
- 680
6
votes
2
answers
447
views
Causal tree v. causal forest - when to use which for HTE?
Would someone be able to explain the considerations for using a causal tree versus a causal forest to estimate heterogeneous treatment effects? Is it that a causal forest is less prone to overfitting? ...
- 193
6
votes
0
answers
176
views
Does Fisher scoring always outperform Newton optimization?
My understanding is that Fisher scoring has several advantages over Newton raphson optimization such as
Computational efficiency: if certain conditions are met (example:During MLE estimation, if link ...
- 313
6
votes
0
answers
660
views
Understanding Object2Vec
AWS released an interesting feature as part of the SageMaker service called Object2Vec that lets you make an embedding for search out of pretty much anything: documents, users, products, ...
- 6,197
6
votes
1
answer
189
views
XGBOOST objective function derivation algebra
I need some help please with the derivation of xgboost objective function. I am following this online tutorial (Math behind GBM and XGBoost)
How do you go from here
$$
loss = \sum_{i=1}^{n} \left( ...
- 593
6
votes
0
answers
737
views
The extrapolation problem: model selection, performance metrics, and improvement
Machine learning models are fit to a response variable within a given range. This leads to weak and sometimes disastrous performance when it comes to instances with an actual response variable outside ...
- 219
6
votes
0
answers
730
views
Why we really need the concept of "Local" Rademacher complexity?
Recently, I have been studying High-Dimensional Statistics: A Non-Asymptotic Viewpoint written by Martin J. Wainwright. In this book, the author uses a special complexity measure which is called Local ...
- 178
6
votes
0
answers
2k
views
Why do we need the gamma parameter in the polynomial kernel of SVMs?
The polynomial kernel is sometimes defined as just:
$$
K(x,y):=(\left<x,y\right>+c)^d
$$
with two parameters: the degree $d$ and constant coefficient $c$.
But others (e.g., libsvm, and sklearn ...
- 3,188
6
votes
0
answers
104
views
Choosing the number of hidden layers and nodes in a Deep Belief Network
What are the recent advances and current best practices in choosing the number and size of stacked Restricted Boltzmann Machines in Deep Belief Networks ?
- 52.1k
6
votes
1
answer
60
views
What type of model can be used to detect changes in periodic behavior?
Imagine we have a data sequence centered around 0 with small fluctuations +/- 1, but approximately every 100 observations it jumps to 10.
If this behavior changed and it started jumping to 5 every 50 ...
- 921
6
votes
0
answers
478
views
Is one-vs-all logit or multionomial logit regression more accurate?
What is advice of when to use one-vs-all logit or multinomial logit regressions? Most importantly, which one has a higher prediction power? Can one test hypothesis and estimate confidence intervals in ...
- 101
6
votes
0
answers
303
views
Sample Space of Machine Learning Classification "Experiment"
If you're trying to classify some input, $\mathbf{x} \in \mathbb{R}^{n}$, to one of $d$ classes using a model with parameters, $\theta$, how are you supposed to think about the experiment of learning ...
- 61