Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

4,265 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
0answers
12 views

Spectral graph convolutional network, re-assigning indices

This is a silly question for whom is familiar with the theory. I came across few papers that use a particular definition of convolution, designed to work with graphs, for example see section 2.1. of ...
3
votes
1answer
38 views

What is the trade off between having a larger validation set versus a smaller one?

Suppose I am comparing several models, e,g, $\{ax\}$, $\{ax+b\}$ and $\{ax^2 + bx + c\}$, $\{ax^3\}$ on data set $\mathcal{D} = \{x_i,y_i\}_{i = 1}^N$ I partition $\mathcal{D}$ into training set ($N-...
3
votes
0answers
134 views

Decomposing R^2 into independent variables

Consider a linear regression model: $$y = β_0 + β_1X_1 + β_2X_2 + ... + β_kX_k + ε$$ where $R^2 = 1 - (SSR/SST)$. I would like to determine the contribution of a factor $i$ (call it $R^2_i$) into ...
3
votes
0answers
46 views

How can statistics be used to avoid “Lending False Credibility To Decisions We've Already Made”

In light of this article Data Science Has Become About Lending False Credibility To Decisions We've Already Made published in Forbes, I would appreciate input from the statistical and data science ...
3
votes
0answers
158 views

Overfitting in Random Forest Classifier?

I would like some help from you in a classification model that I am developing. In summary, the problem is: – Classification problem with binary outcome (0/1) – The classifier is a Random Forest ...
3
votes
1answer
39 views

what is the diffrences between online and one pass learning?

as long as I know, online learning takes actions at each time step (for each data), and one-pass algorithm just can see each data once. I already read Wikipedia: about streaming algorithms. These ...
3
votes
1answer
42 views

k-means clustered data: how to label newly incoming data

I have a data set with labels that were produced by a k-means clustering algorithm. Now there is some data (with the same data structure) from another source and I wonder what is the most sensible way ...
3
votes
0answers
64 views

Analyse sensitivity of hyper-parameters of Machine Learning Models

I want to analyse how sensitive my non neural net machine learning models are to the choice of the different parameters. I am currently using grid search to tune the models. Is there any method that I ...
3
votes
0answers
41 views

Meaning of probability as used in machine learning

In machine learning, it seems that people are often happy to call the output of any function (e.g. logistic function) with the range $[0, 1]$, a probability. How correct is that thinking? How ...
3
votes
0answers
57 views

Interpretation of box-counting method from R

I tried to calculate the fractal dimension of a dataset using the box-counting method with R programming. I used two packages: The first one is fractaldim, ...
3
votes
1answer
70 views

why has author divided by 1.5 in hands on machine learning with scikit learn

I am reading Hands-On Machine Learning with Scikit-Learn and TensorFlow (76/718), and the author is talking about dividing the dataset into a test set which i follow, but then goes on to talk about ...
3
votes
0answers
41 views

Quantifying uncertainty of predictions for new data in the regression tree

I used Regression Learner to train my data. I held out 25% of the input for validation and ran different models for training. Based on the results using RMSE and R-squared, I decided to go for the ...
3
votes
0answers
37 views

Require understanding regarding the concept of restricted estimators

I was reading "The Elements of Statistical Learning Book by Jerome H. Friedman, Robert Tibshirani, and Trevor Hastie" where I encountered the following: The part tells us that the RSS criterion will ...
3
votes
0answers
245 views

Validation ROC AUC not improving with validation cross-entropy loss?

I am training a neural network that is doing binary image classification on several thousand images. I am running 5 fold cross validation (train on 4, validate on 1) with cross entropy (CE) loss. I am ...
3
votes
0answers
42 views

How to incorporate uncertainty and noise information in training and prediction of neural networks?

I am trying to use RNNs to perform state estimation on noisy sensor data. The readings are from a GPS dataset and it provides $[longitude, latitude, n_{satellites}]$. The last column, which is the ...
3
votes
1answer
62 views

Conterfactual estimation in machine learning model

There are various techniques to build counterfactual estimations of certain variables for linear models in observational studies. Some of those are based on comparing the change in the predicted ...
3
votes
1answer
110 views

The violation of triangle inequality in KNN

If the 0<p<1 in the distance metrics, then the triangle inequality is violated. The question as follows Does the violation of this inequality affect the ...
3
votes
0answers
109 views

Imputation and nested cross-validation

I am planning to do a nested cross-validation analysis using regularized regression. The inner loop will be used for model tuning and the outer loop for model assessment (test set). Because some data ...
3
votes
0answers
87 views

How to use MICE in R to fill missing values in test set?

It seems that MICE does not have a "predict" function which allows to use a fitted mids object to predict the missing values in test data set. I can certainly ...
3
votes
0answers
42 views

Confidence intervals of AUC obtained by merging/pooling predictions from different test sets

I have one question regarding the CIs of the AUROC calculated merging/pooling the predictions coming from different test sets. In one analysis, we use a sort of nested cross-validation approach, ...
3
votes
0answers
213 views

Why researchers use conv1d for embeddings instead of dense layers?

In some papers (like Reinforcement learning for Vehicle Routing Problem), researchers use conv1d to embed the problem input into a hyperspace; for example, in solving TSP, they use conv1d on the (x,y) ...
3
votes
0answers
122 views

May too much batch normalization hurt learning?

I was experimenting with some CNN models and reading research material when I realized that it could happen that using only a single batch normalization layer at the early stages of the network could ...
3
votes
0answers
54 views

Generative Adversarial Network and Variational Autoencoders for Independent Component Analysis?

Background: I'm working on a model for independent component analysis (ICA) that is based on a methodology similar to GANs and VAEs. What I'm having trouble understanding is how the choice of the loss ...
3
votes
0answers
37 views

Is Gradiant Boosting a generalization of Adaboost?

I read somewhere that Gradiant boosting is a generalization of Adaboost. However, I cannot see why. Can Anyone elaborate?
3
votes
0answers
297 views

Simple Anomaly Detection Solution

I have a few APIs that are called by clients. I collect data on them such as, what APIs they call, how often they call them etc. So, I have about 6 important metrics. I want to build an anomaly ...
3
votes
0answers
69 views

Binary vs Multiclass Accuracy

Consider an image dataset that has two types of things: cars and airplanes. Let $A$ be a binary classifier trained to classify an image as having a car or airplane. Suppose we now have four refined ...
3
votes
0answers
82 views

A theoretical explanation why ridge is superior to lasso in non-sparse models

There are several posts about the comparison of lasso vs. ridge. However I didn't find an explanation to my question. My question is why ridge is generating lower prediction errors in cases where the ...
3
votes
0answers
88 views

Can someone explain in layman terms how the Contrastive Divergence algorithm works step by step?

I am interested in learning about Restricted Boltzmann Machines (RBMs), but I have trouble understanding how an RBM is trained using contrastive divergence. There are only few papers on this topic and ...
3
votes
0answers
120 views

LASSO: k-fold Cross Validation of AR(p)-process

To improve my intuition on shrinkage models, I want to "recode" the lasso by myself. However, I'm at the point, where I have to program the k-fold Cross-Validation. At my future application of the ...
3
votes
0answers
85 views

LASSO: Difference in selecting tuning parameter for variable selection and prediction purposes

I'm reading Kirkland et al. "LASSO Tuning Parameter Selection" (2015) regarding methods for selecting the tuning Parameter in LASSO regressions. I'm a bit confused about the following Statements. "...
3
votes
1answer
320 views

Trending time series data normalization for Deep Learning

I'm replicating following article Financial Time Series Prediction using Deep Learning and I'm stuck with data normalization. In chapter 5.1 in the second paragraph in the last sentense the authors ...
3
votes
0answers
170 views

Why is bridge regression called “bridge”?

Bridge regression coefficient estimate $\hat{β}^{br}$ are the values that minimize the \begin{equation} \text{RSS} + \lambda \sum_{j=1}^p|\beta_j|^q , \end{equation} where $q \in \mathbb{R}$ and $q &...
3
votes
0answers
325 views

How to handle Zeros in dependent variable in Multiple Linear regression

I am totally new to machine learning (and to this platform too) and was trying to implement Multiple linear Regression to improve my ranking algorithm. I have a data-set which have the following ...
3
votes
0answers
160 views

Number of observations in a node in XGBoost

I understand how the cover is calculated in XGBoost, the sum hessian at that node. For the root node of tree 1 for binary logistic, it becomes n(.5)(1-.5) with base score as 0.5. The cover at root ...
3
votes
2answers
71 views

How to handle machine learning inputs that should be considered as set of vectors, but whoes interpretation is order invariant (order agnostic set)

Basically wondering best practices for input modeling and ML algorithm type(s) for inputs that essentially model samples that are a bag/set of "sub-objects", so order does not matter. Think of the ...
3
votes
0answers
205 views

General PCA optimization problem

I was looking at the PCA optimization problem, which is finding a matrix $U \in \mathbb{R}^{d\times n}$, $n \le d$, that solves the problem $$\max{tr(U^TCU)},\ \ \ s.t. U^TU = I, $$ where $C$ is the ...
3
votes
0answers
84 views

Why use separate trees for each class in multi-class gradient boosting?

Gradient boosted decision trees can be used to solve multi-class classification problems. Friedman (2001) fit $K$ trees on each iteration—one for each class. Multiple GBM implementations also follow ...
3
votes
0answers
53 views

Find Feature Weighting in Deep Learning

If I train a deep neural network on standard tabular data (csv file etc. with labeled features) is there a good way to gauge how important each feature is in a particular new instance's prediction ...
3
votes
0answers
101 views

A few questions regarding the practice of heterogeneous treatment effect analysis (a.k.a, interaction detection or subgroup analysis) methods

Imagine I am looking at a randomized experiment between a control and one or more treatment conditions. For example, I have a treatment that aims to get people out of debt. I randomize people to ...
3
votes
0answers
184 views

Bias Variance decomposition derivation question/proof (from Wikipedia)

I have a question on this derivation of the bias-variance decomposition. At some point they have this part of the expression --> $\mathbf{E}[2y\hat{f}]$ and they say that $\epsilon$ and $\hat{f}$ are ...
3
votes
0answers
343 views

Multinomial Logistic Regreesion with Lasso penalty in R

I am applying regularized logistic regression (in R) to the handwritten digits data set. I have fitted a logistic multinomial model with lasso penalty to the training data. I am asked to obtain the ...
3
votes
1answer
305 views

Combining classification and anomaly detection

I want to build a system, that can classify known classes in a supervised way and at the same time tells if there is a new anomaly class it has not seen before. The user can then label that unknown ...
3
votes
1answer
87 views

Sampling a test set from global spatial data

The basis of testing the accuracy of any machine learning algorithm is to test the trained algorithm on data that it has never seen before. The usual approach to sample the test set is to just ...
3
votes
0answers
81 views

“Data beats hardware and algorithms in neural nets” paper?

I'm trying to track down the citation information for an article. The paper concerned itself with the recent explosion in successful applications of neural networks, and whether this was cause by ...
3
votes
1answer
49 views

Is it valid to calculate propensity score for each treated individual separately?

I have temporal twitter data, and I want to calculate propensity score for the treatment and control group. The problem is, the treatment happened at different time for different user, and I want to ...
3
votes
0answers
842 views

Are XGBoost probabilities well-calibrated?

In general, can you say anything about how well are the probabilities returned by XGBoost are calibrated? Is it true that, because XGBoost directly optimizes log-loss, probabilities are generally well-...
3
votes
0answers
235 views

Voting between classifiers : How to prove it works?

Assume m independent binary classifiers with probability $p$ to be correct $p>0.5$. Show that the probability of a voting, e.g. decision is made by the majority of classifiers is correct with ...
3
votes
0answers
112 views

Assessing correlated predictions

Let's assume we have a prediction algorithm (if it helps, imagine it's using some boosted tree method) that does daily predictions for whether some event will happen to a unit (e.g. a machine that ...
3
votes
0answers
322 views

Position Bias Normalization in CTR Prediction

I am working on Click Through Rate(CTR) prediction model on a toy dataset. The label I am using is #Click/#NumShown. But there is position bias in results shown. ...
3
votes
0answers
316 views

What does the Cholesky decomposition of a correlation matrix tell you?

In this answer, the Cholesky decomposition of a correlation matrix is suggested as the means for testing for multicollinearity. I have a dataset that I am certain has high collinearity. I did the ...