All Questions
20,938 questions
2
votes
1
answer
384
views
Significance test for comparing different 10-fold cross-validated Machine Learning Regressions
Is there a recommended significance test for comparing different 10-fold cross validated regressions?
For instance, I want to compare the performance of LASSO against Random Forest for my dataset.
...
CommunityBot
- 1
21
votes
2
answers
16k
views
The reason of superiority of Limited-memory BFGS over ADAM solver
I am using Multilayer Perceptron MLPClassifier for training a classification model for my problem. I noticed that using the solver lbfgs (I guess it implies Limited-...
2
votes
2
answers
546
views
Learning the Confidence of a Neural Network
Suppose I want to train a deep neural network for classification. The network takes an input vector $x$, and maps this to an output vector $y$. Now, $x$ is of length $n$ and is in fact composed of a ...
CommunityBot
- 1
1
vote
0
answers
37
views
Number of runs needed for Probability of Improvement metric in Deep RL
I'm working with the Probability of Improvement (POI) metric described in [1], Section 4.3.
The paper introduces various aggregate metrics in Section 4.3, and for most of these metrics (IQM, mean, ...
- 654
0
votes
0
answers
8
views
Predicting FPL Player Total Points using Random Forest
I have a dataset with around 100k of gameweek stats in the English Premier League (from 2016-2023). My goal is to predict how many total points a player will score in a certain gameweek/match.
I ...
Mart
0
votes
1
answer
31
views
Fitting a Nonlinear Mixed Model
I’m trying to fit a nonLinear Mixed Model (nLMM) to test whether the abundance of certain organisms was affected by the sampling period after an event that caused a significant increase.
The data show ...
- 473
1
vote
1
answer
21
views
Two questions about the VC theory (on the generalization error bound)
In Andrews Ng's machine learning notes (https://cs229.stanford.edu/main_notes.pdf), he introduced the following bound for the difference between generalization error and training error (see the ...
- 1,335
0
votes
1
answer
230
views
Machine Learning problem - label over multiple lines
Currently I'm trying to work out a project where I would like to recognize movements from videos using machine learning and python.
What I've done so far is extracting the x and y values of body ...
CommunityBot
- 1
1
vote
1
answer
326
views
Unsupervised anomaly detection and classification with event (log) data
I am trying to detect anomalies in a large set of user log events, where most users would be considered “good” and a small minority would be considered “bad.” There are hundreds of event types, which ...
CommunityBot
- 1
5
votes
1
answer
113
views
+50
How to find a de-biased estimator with a ML component in my contaminated data problem?
I am trying to use the output of a machine learning model to estimate (using a maximum likelihood approach) a parameter in a distribution. The estimator I get has a bias which is much larger than the ...
- 101
5
votes
1
answer
115
views
XGBoost/ XGBRanker to produce probabilities instead of ranking scores
I have a dataset of the performance of students in exams which looks like:
...
- 185
3
votes
1
answer
469
views
How is Gini impurity related to accuracy when predicting the majority class?
For simplicity, consider the binary case, where we have a set of elements with each element belonging to one of two classes (0 or 1). Let p(j) be the proportion of ...
- 405
1
vote
0
answers
52
views
Why do machine learning courses on regression mostly focus on gradient descient although we have the closed form estimator $(X'X)^{-1}X'Y$? [duplicate]
In many online machine learning courses and videos(such as Andrew Ng's coursera course), when it comes to regression (for example regressing $Y$ on features $X$), althouth we have the closed form ...
- 67.1k
1
vote
0
answers
32
views
Preprocessing and model selection strategies
I am working on a fault detection problem where each sample is a time series labeled with a specific type of fault. I am using a CNN model and a validation set for hyperparameter tuning. Currently, I ...
- 77
6
votes
2
answers
684
views
Building a Statistically Sound ML Model
Silent reader here in the statistics substack. One thing I've learned is that many "default" machine learning practices are being challenged due to fundamental statistical mistakes. This has ...
- 10.3k
1
vote
1
answer
391
views
Decision Trees and SHAP Values
I've recently been using some (optimal) decision trees methods in R, such as 'evtree' and 'iai.'
Both of these provide really nice interpretable plots. And out of the 12 covariates I have in my model, ...
CommunityBot
- 1
-1
votes
1
answer
76
views
Categorical Dependent Variable
Repost:
Hello all, thank you so much for the response. Here I have provided some information.
a. This is clinical data which is around 859 in sample size.
b. It has 11 columns as input features and ...
- 82.8k
4
votes
3
answers
477
views
Techniques for strategically crafting a ML dataset
For a supervised machine learning application where the input features can be readily calculated and the corresponding labels are the result of a somewhat time-consuming simulation using the inputs, ...
- 82.8k
2
votes
1
answer
332
views
How to use R package caret to build a model and get the internal validation result?
I am a learner of R and machine learning. I don't really understand caret's train function. To make it simple, for example, I want to build a model and get the internal validation result.
At the ...
CommunityBot
- 1
1
vote
2
answers
39
views
How does a single layer/single unit with Adam optimizer network work?
I'm very new to ML and I'm trying to mess around with Linear Regression. I tested sklearn's LinearRegression model and then wanted to compare the results to a very simple neural network.
I created a ...
- 186
27
votes
3
answers
36k
views
How to calculate out of sample R squared?
I know this probably has been discussed somewhere else, but I have not been able to find an explicit answer. I am trying to use the formula $R^2 = 1 - SSR/SST$ to calculate out-of-sample $R^2$ of a ...
- 11
0
votes
1
answer
262
views
Object distortion after ROI Align in Mask R-CNN
In Mask R-CNN, if there are 2 proposed ROIs which cover 2 objects that looks like below:
#1 A square object
#2 A rectangular object
So my question is:
After ROI Align, is the #2 feature map ...
CommunityBot
- 1
2
votes
1
answer
37
views
Handling a very informative feature with significant missing values
I have a machine learning model where the goal is prediction in the context of regression.
For my metric of interest, there is a feature which is extremely informative but has significant missing ...
- 1,335
4
votes
1
answer
1k
views
m out of n bootstrap implementation in R
I am wishing to estimate the sampling distribution of an extreme order statistic (the sample maximum). The usual nonparametric (n-out-of-n) bootstrap fails miserably in this case.
Chernick (2011) ...
- 5,730
0
votes
1
answer
231
views
How to compute Upper Confidence Bound Properly In Multiarmed Bandit Problem
I'm currently working on implementing the Upper Confidence Bound (UCB) algorithm for the Multiarmed Bandit Problem, but I'm encountering some difficulties with the computation. Here's what I've ...
CommunityBot
- 1
0
votes
1
answer
7k
views
How to design a many-to-many LSTM RNN in Keras
I have timeseries data with 1 minute cadence with 4 features, and I want to try to predict the time-evolution of 2 of these features using a RNN using LSTMs in Keras. My aim is to predict the e.g. ...
CommunityBot
- 1
4
votes
2
answers
8k
views
Missing data in k-means cluster model
I'm working on clustering email addresses using K-means based on their value to and engagement with the company (metrics such as % of emails opened, # of web browsing sessions, etc). I would like to ...
CommunityBot
- 1
5
votes
2
answers
732
views
Calculating CI's using bootstrapping on the holdout test dataset
I’m trying to calculate 95% confidence intervals for the sensitivity and specificity of a decision model that I’m building.
I’ve split my dataset into 90/10 train and test sets. I’ve used the 90% ...
CommunityBot
- 1
1
vote
1
answer
301
views
Neural Networks - Can I Use Any Activation for the Output Layer?
I'm new to neural networks, and in almost everything I'm reading, the activation function recommended on the output layer follows a specific pattern:
If the network does binary classification (1 ...
CommunityBot
- 1
0
votes
1
answer
364
views
Classification with partially labelled data - potential positives
I am having trouble figuring out the best approach for a classification problem:
My data:
For each physician in my data, I have a feature set of every different medical procedure where the feature ...
CommunityBot
- 1
1
vote
3
answers
680
views
Unsupervised Time series anomaly detection
I have 3D printer that working exactly 400 second for printing element X [0-400].
The printer produce 30 signals (features like VOLT,X,Y,Z,TEMP etc') in frequency of 50HZ (every sample 0.02 ms) ,for ...
CommunityBot
- 1
4
votes
1
answer
485
views
How to initialize and train a Hidden Markov Model to improve the classification produced by a previous classifier?
Say we've previously used a neural network or some other classifier C with $N$ training samples $I:=\{I_1,...I_N\}$ (that has a sequence or context, but is ignored by C) the, belonging to $K$ classes. ...
CommunityBot
- 1
1
vote
2
answers
2k
views
How to Resolve Variational Autoencoder (VAE) Model Collapse in Reconstruction Task Using Sensor Data?
I am currently experiencing a suspected model collapse in a Variational Autoencoder (VAE) model I am working with. Below are details on the project setup and the issue at hand:
Project Goal: Exploring ...
CommunityBot
- 1
1
vote
1
answer
261
views
Loss Function for Binary Classification with Multiple Correct Choices
I have a binary classification problem, where there are multiple correct predictions, however, I would consider the prediction to be correct if the highest confidence prediction of a 1 is correct.
I ...
CommunityBot
- 1
8
votes
2
answers
3k
views
Does it make sense to regularize the loss function for binary/multi-class classification?
When discussing linear regression it is well known that you can add regularization terms, such as,
$$\lambda \|w\|^2 \quad \text{(Tikhonov regularization)}$$
to the empirical error/loss function.
...
0
votes
1
answer
412
views
How can I replicate the process sklearn calculates the posterior probabilities?
I have a question pertaining to scikit-learn methods.
Can I get the same probabilities obtained with predict_log_proba() by hand calculating the likelihoods and prior obtained with feature_log_prob_ ...
CommunityBot
- 1
3
votes
2
answers
777
views
Data leakage when using walk forward optimization
I am setting up a neural network that will predict the incoming customers at a store for the next seven days (the output is a list with seven numbers, one for each day). As input, I will give the ...
- 11
6
votes
1
answer
170
views
Why the loss is not considered as a "supervisory signal" in unsupervised learning?
It is said that supervised is different from unsupervised learning due to the presence of "supervisory signals" aka labels.
However, in both cases we have a loss function. Isn't the loss a ...
- 1,335
0
votes
0
answers
15
views
Comparing probabilities of two models
Consider a dataset and two binary classes CLASS_A and CLASS_B, with different proportions of 0 and 1. Suppose we train a model such as XGBClassifier for both ...
- 1,690
2
votes
1
answer
460
views
How to prove that a function is 2-increasing (copula)
There are three conditions to prove that a function is a copula:
$C(u,0)=0=C(0,v)$ grounded.
$C(u,1)= u, C(1,v)= v$.
$C(u,v)$ 2-increasing function.
Here I am concerning in the last condition how to ...
CommunityBot
- 1
1
vote
1
answer
382
views
Prediction model with constraint / penalty
I am attempting to predict rental price (Sq/ft) for a Retail space. I have a vector of demand / economic variables and other control variables such location and time fixed effects. I'd like to add a ...
CommunityBot
- 1
2
votes
1
answer
1k
views
What is the order when doing feature engineering? (imputation, encoding, etc.) [closed]
I am self learning machine learning right now, and I am confused with what should I do first.
Should I impute the missing value before encoding the categorical variable?
Also, I am learning from ...
CommunityBot
- 1
1
vote
1
answer
469
views
When to use synthetic data and when to use regularization parameters to avoid the over fitting and which is better?
Can anyone explain me when to consider generating the synthetic data or when to consider regularization parameters to reduce the error so the machine learning model will not overfit
CommunityBot
- 1
1
vote
1
answer
397
views
Ads Click Through Prediction without Test Dataset
I recently counter an assignment which is to predict whether users will click ads shown on some websites in the next week based on users' log history. The log contains users' id, os and browser type, ...
CommunityBot
- 1
0
votes
0
answers
8
views
Statistical Testing with Minimal Samples for Reinforcement Learning Algorithms
I'm working on comparing two reinforcement learning algorithms where:
Running experiments is extremely computationally expensive
Based on preliminary results, Algorithm B consistently and
...
- 654
1
vote
1
answer
228
views
Weak Supervision - training generative model without knowing the true label
Recently I've been reading about weak supervision. I understand most of the concept details, there's one thing that is not clear to me though.
In the generative model part (creating generative model ...
CommunityBot
- 1
1
vote
1
answer
918
views
Calculate the output of a Neural Network
I have the following problem:
Here is my approach:
With the activation function: $F(x) = x^2 + 2x + 3$, we can calculate the activation of the two units of the second layer by: $a_1^2 = F(w_{13}\...
CommunityBot
- 1
2
votes
1
answer
374
views
Gini impurity greedily optimises a loss function in decision trees
I am trying to understand how the Gini criterion for decision decision tree construction actually greedily optimises a loss function.
The Gini impurity, sometimes also called Gini index, for a region (...
CommunityBot
- 1
8
votes
4
answers
560
views
Signal-to-noise ratio in predictive modeling and machine learning
The interesting comments to this question get into how signal-to-noise ratio plays into ability to make predictions. Being more explicit about it, how does signal-to-noise ratio factor into how good ...
- 171
12
votes
7
answers
3k
views
Is the mean of samples still a valid sample?
Suppose I sample $n$ times from a distribution
$$
x_1, \ldots, x_n \sim p_\theta(x)
$$
is the mean of the samples always a valid sample from the target distribution? I.e. is $\overline{x}$ a valid ...
- 11.5k