Questions tagged [machine-learning]

Machine learning algorithms build a model of the training data. The term "machine learning" is vaguely defined; it includes what is also called statistical learning, reinforcement learning, unsupervised learning, etc. ALWAYS ADD A MORE SPECIFIC TAG.

4
votes
0answers
304 views

Category selection for text classification

It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
3
votes
0answers
53 views

Does anybody know this measure of model fit / prediction error?

Let $y_i$ be the true value and $\hat{y}_i$ a prediction from a model. Then, for example $$B=n^{-1}\sum_{i=1}^n \hat{y}_i - y_i$$ is the prediction bias and $$MSE=n^{-1}\sum_{i=1}^n (\hat{y}_i - y_i)^...
3
votes
0answers
12 views

Spectral graph convolutional network, re-assigning indices

This is a silly question for whom is familiar with the theory. I came across few papers that use a particular definition of convolution, designed to work with graphs, for example see section 2.1. of ...
3
votes
0answers
109 views

Decomposing R^2 into independent variables

Consider a linear regression model: $$y = β_0 + β_1X_1 + β_2X_2 + ... + β_kX_k + ε$$ where $R^2 = 1 - (SSR/SST)$. I would like to determine the contribution of a factor $i$ (call it $R^2_i$) into ...
3
votes
0answers
45 views

How can statistics be used to avoid “Lending False Credibility To Decisions We've Already Made”

In light of this article Data Science Has Become About Lending False Credibility To Decisions We've Already Made published in Forbes, I would appreciate input from the statistical and data science ...
3
votes
0answers
131 views

Overfitting in Random Forest Classifier?

I would like some help from you in a classification model that I am developing. In summary, the problem is: – Classification problem with binary outcome (0/1) – The classifier is a Random Forest ...
3
votes
0answers
57 views

Analyse sensitivity of hyper-parameters of Machine Learning Models

I want to analyse how sensitive my non neural net machine learning models are to the choice of the different parameters. I am currently using grid search to tune the models. Is there any method that I ...
3
votes
0answers
40 views

Meaning of probability as used in machine learning

In machine learning, it seems that people are often happy to call the output of any function (e.g. logistic function) with the range $[0, 1]$, a probability. How correct is that thinking? How ...
3
votes
0answers
54 views

Interpretation of box-counting method from R

I tried to calculate the fractal dimension of a dataset using the box-counting method with R programming. I used two packages: The first one is fractaldim, ...
3
votes
0answers
40 views

Quantifying uncertainty of predictions for new data in the regression tree

I used Regression Learner to train my data. I held out 25% of the input for validation and ran different models for training. Based on the results using RMSE and R-squared, I decided to go for the ...
3
votes
0answers
37 views

Require understanding regarding the concept of restricted estimators

I was reading "The Elements of Statistical Learning Book by Jerome H. Friedman, Robert Tibshirani, and Trevor Hastie" where I encountered the following: The part tells us that the RSS criterion will ...
3
votes
0answers
214 views

Validation ROC AUC not improving with validation cross-entropy loss?

I am training a neural network that is doing binary image classification on several thousand images. I am running 5 fold cross validation (train on 4, validate on 1) with cross entropy (CE) loss. I am ...
3
votes
0answers
38 views

How to incorporate uncertainty and noise information in training and prediction of neural networks?

I am trying to use RNNs to perform state estimation on noisy sensor data. The readings are from a GPS dataset and it provides $[longitude, latitude, n_{satellites}]$. The last column, which is the ...
3
votes
0answers
103 views

Imputation and nested cross-validation

I am planning to do a nested cross-validation analysis using regularized regression. The inner loop will be used for model tuning and the outer loop for model assessment (test set). Because some data ...
3
votes
0answers
86 views

How to use MICE in R to fill missing values in test set?

It seems that MICE does not have a "predict" function which allows to use a fitted mids object to predict the missing values in test data set. I can certainly ...
3
votes
0answers
41 views

Confidence intervals of AUC obtained by merging/pooling predictions from different test sets

I have one question regarding the CIs of the AUROC calculated merging/pooling the predictions coming from different test sets. In one analysis, we use a sort of nested cross-validation approach, ...
3
votes
0answers
207 views

Why researchers use conv1d for embeddings instead of dense layers?

In some papers (like Reinforcement learning for Vehicle Routing Problem), researchers use conv1d to embed the problem input into a hyperspace; for example, in solving TSP, they use conv1d on the (x,y) ...
3
votes
0answers
114 views

May too much batch normalization hurt learning?

I was experimenting with some CNN models and reading research material when I realized that it could happen that using only a single batch normalization layer at the early stages of the network could ...
3
votes
0answers
53 views

Generative Adversarial Network and Variational Autoencoders for Independent Component Analysis?

Background: I'm working on a model for independent component analysis (ICA) that is based on a methodology similar to GANs and VAEs. What I'm having trouble understanding is how the choice of the loss ...
3
votes
0answers
36 views

Is Gradiant Boosting a generalization of Adaboost?

I read somewhere that Gradiant boosting is a generalization of Adaboost. However, I cannot see why. Can Anyone elaborate?
3
votes
0answers
274 views

Simple Anomaly Detection Solution

I have a few APIs that are called by clients. I collect data on them such as, what APIs they call, how often they call them etc. So, I have about 6 important metrics. I want to build an anomaly ...
3
votes
0answers
68 views

Binary vs Multiclass Accuracy

Consider an image dataset that has two types of things: cars and airplanes. Let $A$ be a binary classifier trained to classify an image as having a car or airplane. Suppose we now have four refined ...
3
votes
0answers
82 views

A theoretical explanation why ridge is superior to lasso in non-sparse models

There are several posts about the comparison of lasso vs. ridge. However I didn't find an explanation to my question. My question is why ridge is generating lower prediction errors in cases where the ...
3
votes
0answers
88 views

Can someone explain in layman terms how the Contrastive Divergence algorithm works step by step?

I am interested in learning about Restricted Boltzmann Machines (RBMs), but I have trouble understanding how an RBM is trained using contrastive divergence. There are only few papers on this topic and ...
3
votes
0answers
118 views

LASSO: k-fold Cross Validation of AR(p)-process

To improve my intuition on shrinkage models, I want to "recode" the lasso by myself. However, I'm at the point, where I have to program the k-fold Cross-Validation. At my future application of the ...
3
votes
0answers
85 views

LASSO: Difference in selecting tuning parameter for variable selection and prediction purposes

I'm reading Kirkland et al. "LASSO Tuning Parameter Selection" (2015) regarding methods for selecting the tuning Parameter in LASSO regressions. I'm a bit confused about the following Statements. "...
3
votes
0answers
164 views

Why is bridge regression called “bridge”?

Bridge regression coefficient estimate $\hat{β}^{br}$ are the values that minimize the \begin{equation} \text{RSS} + \lambda \sum_{j=1}^p|\beta_j|^q , \end{equation} where $q \in \mathbb{R}$ and $q &...
3
votes
0answers
309 views

How to handle Zeros in dependent variable in Multiple Linear regression

I am totally new to machine learning (and to this platform too) and was trying to implement Multiple linear Regression to improve my ranking algorithm. I have a data-set which have the following ...
3
votes
0answers
150 views

Number of observations in a node in XGBoost

I understand how the cover is calculated in XGBoost, the sum hessian at that node. For the root node of tree 1 for binary logistic, it becomes n(.5)(1-.5) with base score as 0.5. The cover at root ...
3
votes
0answers
202 views

General PCA optimization problem

I was looking at the PCA optimization problem, which is finding a matrix $U \in \mathbb{R}^{d\times n}$, $n \le d$, that solves the problem $$\max{tr(U^TCU)},\ \ \ s.t. U^TU = I, $$ where $C$ is the ...
3
votes
0answers
83 views

Why use separate trees for each class in multi-class gradient boosting?

Gradient boosted decision trees can be used to solve multi-class classification problems. Friedman (2001) fit $K$ trees on each iteration—one for each class. Multiple GBM implementations also follow ...
3
votes
0answers
53 views

Find Feature Weighting in Deep Learning

If I train a deep neural network on standard tabular data (csv file etc. with labeled features) is there a good way to gauge how important each feature is in a particular new instance's prediction ...
3
votes
0answers
99 views

A few questions regarding the practice of heterogeneous treatment effect analysis (a.k.a, interaction detection or subgroup analysis) methods

Imagine I am looking at a randomized experiment between a control and one or more treatment conditions. For example, I have a treatment that aims to get people out of debt. I randomize people to ...
3
votes
0answers
183 views

Bias Variance decomposition derivation question/proof (from Wikipedia)

I have a question on this derivation of the bias-variance decomposition. At some point they have this part of the expression --> $\mathbf{E}[2y\hat{f}]$ and they say that $\epsilon$ and $\hat{f}$ are ...
3
votes
0answers
339 views

Multinomial Logistic Regreesion with Lasso penalty in R

I am applying regularized logistic regression (in R) to the handwritten digits data set. I have fitted a logistic multinomial model with lasso penalty to the training data. I am asked to obtain the ...
3
votes
0answers
81 views

“Data beats hardware and algorithms in neural nets” paper?

I'm trying to track down the citation information for an article. The paper concerned itself with the recent explosion in successful applications of neural networks, and whether this was cause by ...
3
votes
0answers
812 views

Are XGBoost probabilities well-calibrated?

In general, can you say anything about how well are the probabilities returned by XGBoost are calibrated? Is it true that, because XGBoost directly optimizes log-loss, probabilities are generally well-...
3
votes
0answers
232 views

Voting between classifiers : How to prove it works?

Assume m independent binary classifiers with probability $p$ to be correct $p>0.5$. Show that the probability of a voting, e.g. decision is made by the majority of classifiers is correct with ...
3
votes
0answers
111 views

Assessing correlated predictions

Let's assume we have a prediction algorithm (if it helps, imagine it's using some boosted tree method) that does daily predictions for whether some event will happen to a unit (e.g. a machine that ...
3
votes
0answers
308 views

Position Bias Normalization in CTR Prediction

I am working on Click Through Rate(CTR) prediction model on a toy dataset. The label I am using is #Click/#NumShown. But there is position bias in results shown. ...
3
votes
0answers
310 views

What does the Cholesky decomposition of a correlation matrix tell you?

In this answer, the Cholesky decomposition of a correlation matrix is suggested as the means for testing for multicollinearity. I have a dataset that I am certain has high collinearity. I did the ...
3
votes
0answers
86 views

State of the art feature extractor from text for machine learning

In machine learning for text classification, the first step after acquiring and cleaning the data is that of feature extraction. Since computers can't understand language like humans, the language ...
3
votes
0answers
349 views

Using confusion matrix to improve my SVM

I ran an SVM classifier on the CIFAR_10 classification workbench. I got about 2/3 accuracy (which I think is Ok, but I want to improve...) Here is my confusion matrix: ...
3
votes
0answers
77 views

Deriving bias of local linear regression

I have been reading Elements of Statistical Learning, and in Chapter 6 on Local Regression, they state the following for fitting local regression at point $x_0$ from data $\mathbf{x}$ of size $\rm{dim}...
3
votes
0answers
607 views

When, if at all, to reset the state of an LSTM when training and when testing?

I am building an LSTM that takes in time-series financial data. My dataset is made up of IDs (each ID is a certain stock), and timestamps. For each ID at each timestamp, there are a number of features ...
3
votes
0answers
415 views

How does choosing between pre and post zero padding of sequences impact results

I'm working on an NLP sequence labelling problem. My data consists of variable length sequences (w_1, w_2, ..., w_k) with corresponding labels ...
3
votes
0answers
131 views

Model training: difference between inference, marginalisation, and estimation

I am working through the lecture slides of Carl Rasmussen's Probablistic Machine Learning course. In slide 6 of the first lecture it lists a number of ways that one can learn the parameters (A, C) and ...
3
votes
0answers
64 views

What are some machine learning problems that can be attacked with continuous multiobjective optimization?

I am working on continuous vector optimization, and hence continuous multiobjective optimization is a particular case. I am interested in finding applications in machine learning for this problems. Is ...
3
votes
0answers
219 views

Expectation of Covariance Matrix for Deep Gaussian Processes

I am currently reading the paper entitled "Variational Auto-Encoded Deep Gaussian Processes" by Dai et al, a copy of which may be found here. The paper proposes a stacking of Gaussian Process Latent ...
3
votes
0answers
250 views

Quiz: Determine first principal component from data-plots

We see four data plots. The goal: How does the first principal component look for each plot a-d. For plot d, it is true that both clusters have same number of datapoints. First principal component ...