Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

5,909 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
5
votes
0answers
163 views

A few questions regarding the practice of heterogeneous treatment effect analysis (a.k.a, interaction detection or subgroup analysis) methods

Imagine I am looking at a randomized experiment between a control and one or more treatment conditions. For example, I have a treatment that aims to get people out of debt. I randomize people to ...
5
votes
0answers
219 views

Quantifying uncertainty when fitting a statistical model to partial effects/dependencies from a random forest (or other machine learning model)

Question: I estimate the partial dependence of $y$ on one predictor in a fitted random forest (RF). I want to now fit a parametric model to this partial dependence. How can I estimate my uncertainty ...
5
votes
0answers
727 views

Why no one talks about stochastic conjugate gradient descent?

As is known to all, stochastic gradient descent is a popular optimizer in machine learning. There have been many variants of SGD. However, it has come to my attention that no one talks about the ...
5
votes
2answers
882 views

First-layer Visualizations in a neural network

I am reading the lectures on "Convolutional Neural Networks for Visual Recognition", and in this lecture they deal with first layer visualization. As you can see in the figure below- this figure ...
5
votes
0answers
1k views

How to include negative examples in multi-class classification?

I have a problem similar to this question: How do I use negative examples (in addition to positive ones) for training a multiclass softmax classifier (or a neural net with softmax output)? where I ...
5
votes
2answers
3k views

Observations weight-age in a Machine Learning model

I want to know is there any way in R/Python to specify to the model to emphasize its learning more on specific subset of data , while it considers the whole data. For example - i have sales behavior ...
5
votes
1answer
513 views

Measuring the bias-variance tradeoff

Does anyone know of a metric that quantifies the bias-variance tradeoff of a given fitted model? I'm not talking about measuring the MSE in cross validation, I'm interested in a single generic or ...
5
votes
0answers
106 views

Sparse vs compact representations

In sparse representations, we like to find representation of the input where most elements are nearly zero. On the other hand, in some applications we prefer dense representations such as word ...
5
votes
0answers
392 views

Unsupervised Anomaly Detection Threshold Selection

If we have a data set that contains only positive examples I am wondering how we can effectively choose a threshold for an anomaly detection technique. Are there anomaly detection techniques that can ...
5
votes
1answer
2k views

Using Rolling Forecast Origin Resampling in R for Neural Network Time Series

I am new to time series prediction and forecasting with neural networks and am having trouble with cross validation. I am fitting a multivariate time series. I have 236 monthly observations. I am ...
5
votes
0answers
272 views

Sample Space of Machine Learning Classification “Experiment”

If you're trying to classify some input, $\mathbf{x} \in \mathbb{R}^{n}$, to one of $d$ classes using a model with parameters, $\theta$, how are you supposed to think about the experiment of learning ...
5
votes
0answers
356 views

Is there an appropriate order to apply bagging and filter feature selection?

I'm training a (regression) learner on a $p \gg n$ problem, including bagging and filter feature selection (information gain). I'm in doubt though regarding the order of the procedures: Apply the ...
5
votes
0answers
11k views

True positive, false negative, true negative, false positive definitions for multiclass-multilabel classification?

I'm trying to apply some evaluation metrics to several clustering methods. I thought that I knew them basing on the multiclass confusion matrix, considering the rows as the actual classes and the ...
5
votes
0answers
1k views

Recurrent neural network for object tracking & position filtering?

Would a recurrent neural network be appropriate for object tracking tasks? Mainly I will have 3D feature vectors $(x, y, t)$ where $x$ and $y$ are the positions of an object in the image and $t$ is ...
5
votes
0answers
363 views

Interpreting hidden layer representations in ANNs

I'm using the fann library for writing an Artificial Neural Network in C++. I trained my network for the task of recognizing faces inside a set of 128x128 .png images, using three different algorithms:...
5
votes
0answers
409 views

Ideal statistical or machine learning technique to model highly cross-correlated data

I'm trying to build a model that can predict streamflow for an alpine (snowmelt-fed) watershed using snow albedo (roughly, the energy reflectance of the snow) data. Albedo controls the melt of the ...
5
votes
0answers
366 views

What enforces features diversity in RBM?

I'm working on an implementation of a Restricted Boltzman Machine (RBM). I made some tests on the MNIST dataset trying to learn a representation of the digit 2. My inputs are binary images. My aim is ...
5
votes
0answers
544 views

Basic idea of zero inflated two part models(hurdel) and application to big data (machine learning)

I'm currently working on the data which has 90% 0s in response variable. Based on my research, it seems zero inflated models could be a solution to this. However, while I was reading related documents,...
5
votes
0answers
1k views

What are appropriate validation methods for a Bayesian network model with low sample size?

I am currently using a Bayesian network model with 20 variables and 210 data points, with 15 locations measured at 14 different time points each. There are also some restrictions on what types of ...
5
votes
0answers
467 views

Rademacher bounds for unbounded loss functions

All common treatment of PAC bounds based on Rademacher complexity assume a bounded loss function (for a self-contained treatemnt, see this handout by Schapire. However, I could not find any result for ...
5
votes
0answers
153 views

Compressed sensing: Optimization in $L_1$ norm and total variation with fourier coefficients

I'm reading the article Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information (Candes, Romberg and Tao, 2004). In this article they are talking ...
5
votes
0answers
4k views

Minimum training sample size required for a classifier

What is the best method to determine the minimum number of training samples required for a classifier? I am only comparing one classifier (four class problem), discriminant function analysis (DFA) ...
5
votes
0answers
186 views

Asynchronous data stream matching

Suppose you have a classifier $C^n$ which continuously outputs a stream of classification labels $K^n_i$ and corresponding timestamps $T^n_i$. Also, we know the prior probability $P(K^n) \forall n$. ...
5
votes
0answers
2k views

Why would concatenating feature vectors lead to better estimates?

I wish to estimate the state of a system from two separate and disparate observations. A simple approach that I have seen in some literature is to combine the feature vectors (observations) by simply ...
5
votes
1answer
882 views

Bayesian hyperparameter optimization + cross-validation

I want to use Bayesian optimization to search a space of hyperparameters for a neural network model. My objective function for this optimization is validation set accuracy. In addition, I want to ...
5
votes
1answer
714 views

Unlearning Neural Network? Prevent learning from a specific feature

Is it possible to train a NN to avoid the features that a different neural network finds? For example, let's train a simple 1 layer CNN with 1x1 kernels on a supervised binary classification problem. ...
4
votes
0answers
56 views

Derivation of k nearest neighbor classification rule

One way to derive the k-NN decision rule based on the k-NN density estimation goes as follows: given $k$ the number of neighbors, $k_i$ the number of neighbors of class $i$ in the bucket, $N$ the ...
4
votes
1answer
270 views

Why are the tied weights in autoencoders transposed and not inverted?

I am currently reading about Autoencoders. From what I understand so far, when we are dealing with a symmetrical autoencoder, a good practice is to tie the weights of the decoder layers to the weights ...
4
votes
0answers
79 views

ANOVA vs. mixed models

I'm confused between the differences between x-way/mixed ANOVA models and mixed models. Is there a difference? If so, what is the difference and why?
4
votes
0answers
43 views

How can I prove the ill-effects of binning/discretization?

There is a binary classification model built where there is grouping of continuous variables into arbitrary ranges which I am told is to include a good number of outliers in the data set. How do I ...
4
votes
2answers
356 views

Reconstruction Error: Principal component analysis vs Probabilistic prinicpal component analysis

I am working through the book "Machine Learning: A Probabilistic Perspective". After introducing PCA and Probabilistic PCA, the following graphic is shown (the upper two graphics ...
4
votes
0answers
87 views

Why is BART so accurate in causal inference?

The famous paper Dorie,2017 shows that BART performs dramatically well in causal inference. In my replication, MSE in BART can be 40% lower than MSE in other machine learning methods. But all machine ...
4
votes
0answers
49 views

Why is homoscedasticity (homogeneity of variance) important in neural network layers?

I'm studying the famous Xavier initialization paper (Understanding the Difficulty of Training Deep Feedforward Neural Networks (Glorot and Bengio, 2010)) and had a question. When they explain the ...
4
votes
0answers
45 views

Does Fisher scoring always outperform Newton optimization?

My understanding is that Fisher scoring has several advantages over Newton raphson optimization such as Computational efficiency: if certain conditions are met (example:During MLE estimation, if link ...
4
votes
0answers
57 views

Why is Rademacher complexity defined the way it is?

For reference, this is the definition of empirical Rademacher complexity from Foundations of Machine Learning (page 30): Let $\mathcal{G}$ be a family of functions mapping from $\mathcal{Z}$ to $[...
4
votes
0answers
83 views

How to explain random forest ML algorithm doesn't learn at all, while logistic regression learns very well?

My prediction task is as follows: Use name to predict people's ethnicity (into 4 categories: "English", "French", "Chinese", and "All others") as a multiclass classification problem. The name ...
4
votes
0answers
44 views

How is EMSE derived for causal trees in Athey and Imbens (PNAS 2016)?

Athey and Imbens build a non-parametric matching procedure to identify and estimate causal effects. To this end, they minimize the expected mean squared error (EMSE) of their procedure, but I don't ...
4
votes
1answer
104 views

Workaround for word embeddings that do not “see” antonyms

Most word embeddings do not “see” antonyms. For instance, among many words they will place vectors for “dependent” and “independent” (as an example) quite close, — actually as close as with synonyms ...
4
votes
1answer
67 views

“row” and “column” are the names of axes of 2d array, is there a similar naming for a 3d array?

row and column are the names of axes of 2d array. this python array, array([[0, 1, 2], [3, 4, 5], [6, 7, 8]]) could be viewed as a matrix that ...
4
votes
0answers
302 views

The extrapolation problem: model selection, performance metrics, and improvement

Machine learning models are fit to a response variable within a given range. This leads to weak and sometimes disastrous performance when it comes to instances with an actual response variable outside ...
4
votes
0answers
119 views

How to model probabilistic inputs with continuous output using regression

I have trained a multi-output classifier that takes an image as input and returns softmax logits as output. To be specific, the multi-output classifier takes an image and says the probability that ...
4
votes
1answer
123 views

Why do we use the average-pooling layers instead of max-pooling layers in the DenseNet?

In the paper of DenseNet. The author adopted average-pooling in the transition layers. So what is the motivation of such choice? Why not using max-pooling layers?
4
votes
0answers
67 views

How to reconstruct an image from a training set?

Description: I have taken a series of images/photos of a panorama from different positions (x,y) in space pretty close to each other (max 100m difference). Here there is a top view representation to ...
4
votes
1answer
130 views

Proof for asymptotic error in logistic regression

Ng, A.Y., and Jordan, M.I. (2001). On Discriminative vs. Generative classifiers: A comparison of logistic regression and naive Bayes. Advances in Neural Information Processing Systems, 14, pp. 841-8, ...
4
votes
0answers
60 views

How can statistics be used to avoid “Lending False Credibility To Decisions We've Already Made”

In light of this article Data Science Has Become About Lending False Credibility To Decisions We've Already Made published in Forbes, I would appreciate input from the statistical and data science ...
4
votes
0answers
1k views

Overfitting in Random Forest Classifier?

I would like some help from you in a classification model that I am developing. In summary, the problem is: – Classification problem with binary outcome (0/1) – The classifier is a Random Forest ...
4
votes
0answers
510 views

Analyse sensitivity of hyper-parameters of Machine Learning Models

I want to analyse how sensitive my non neural net machine learning models are to the choice of the different parameters. I am currently using grid search to tune the models. Is there any method that I ...
4
votes
0answers
260 views

PCA, SMOTE and cross validation - how to combine them together?

I was reading a lot recently about PCA and cross validation and it seems that the majority call it malpractice to do PCA before cross validation. I would also like to perform SMOTE, but there is a ...
4
votes
0answers
115 views

random kitchen sinks as approximation to kernel machine

In the paper Rahimi, Ali, and Benjamin Recht. "Random features for large-scale kernel machines." Advances in neural information processing systems. 2008. the author introduces a way to ...
4
votes
0answers
1k views

Why do we need the gamma parameter in the polynomial kernel of SVMs?

The polynomial kernel is sometimes defined as just: $$ K(x,y):=(\left<x,y\right>+c)^d $$ with two parameters: the degree $d$ and constant coefficient $c$. But others (e.g., libsvm, and sklearn ...