All Questions
20,938 questions
1
vote
0
answers
35
views
Number of runs needed for Probability of Improvement metric in Deep RL
I'm working with the Probability of Improvement (POI) metric described in [1], Section 4.3.
The paper introduces various aggregate metrics in Section 4.3, and for most of these metrics (IQM, mean, ...
- 654
0
votes
0
answers
6
views
Predicting FPL Player Total Points using Random Forest
I have a dataset with around 100k of gameweek stats in the English Premier League (from 2016-2023). My goal is to predict how many total points a player will score in a certain gameweek/match.
I ...
Mart
0
votes
1
answer
30
views
Fitting a Nonlinear Mixed Model
I’m trying to fit a nonLinear Mixed Model (nLMM) to test whether the abundance of certain organisms was affected by the sampling period after an event that caused a significant increase.
The data show ...
- 33
5
votes
1
answer
115
views
XGBoost/ XGBRanker to produce probabilities instead of ranking scores
I have a dataset of the performance of students in exams which looks like:
...
- 185
1
vote
1
answer
21
views
Two questions about the VC theory (on the generalization error bound)
In Andrews Ng's machine learning notes (https://cs229.stanford.edu/main_notes.pdf), he introduced the following bound for the difference between generalization error and training error (see the ...
- 3,050
1
vote
0
answers
52
views
Why do machine learning courses on regression mostly focus on gradient descient although we have the closed form estimator $(X'X)^{-1}X'Y$? [duplicate]
In many online machine learning courses and videos(such as Andrew Ng's coursera course), when it comes to regression (for example regressing $Y$ on features $X$), althouth we have the closed form ...
- 3,050
1
vote
0
answers
32
views
Preprocessing and model selection strategies
I am working on a fault detection problem where each sample is a time series labeled with a specific type of fault. I am using a CNN model and a validation set for hyperparameter tuning. Currently, I ...
- 77
5
votes
1
answer
111
views
+50
How to find a de-biased estimator with a ML component in my contaminated data problem?
I am trying to use the output of a machine learning model to estimate (using a maximum likelihood approach) a parameter in a distribution. The estimator I get has a bias which is much larger than the ...
- 101
6
votes
2
answers
682
views
Building a Statistically Sound ML Model
Silent reader here in the statistics substack. One thing I've learned is that many "default" machine learning practices are being challenged due to fundamental statistical mistakes. This has ...
2
votes
1
answer
37
views
Handling a very informative feature with significant missing values
I have a machine learning model where the goal is prediction in the context of regression.
For my metric of interest, there is a feature which is extremely informative but has significant missing ...
- 498
1
vote
2
answers
39
views
How does a single layer/single unit with Adam optimizer network work?
I'm very new to ML and I'm trying to mess around with Linear Regression. I tested sklearn's LinearRegression model and then wanted to compare the results to a very simple neural network.
I created a ...
- 111
6
votes
1
answer
170
views
Why the loss is not considered as a "supervisory signal" in unsupervised learning?
It is said that supervised is different from unsupervised learning due to the presence of "supervisory signals" aka labels.
However, in both cases we have a loss function. Isn't the loss a ...
- 641
0
votes
0
answers
15
views
Comparing probabilities of two models
Consider a dataset and two binary classes CLASS_A and CLASS_B, with different proportions of 0 and 1. Suppose we train a model such as XGBClassifier for both ...
- 1,690
0
votes
0
answers
8
views
Statistical Testing with Minimal Samples for Reinforcement Learning Algorithms
I'm working on comparing two reinforcement learning algorithms where:
Running experiments is extremely computationally expensive
Based on preliminary results, Algorithm B consistently and
...
- 654
-1
votes
1
answer
76
views
Categorical Dependent Variable
Repost:
Hello all, thank you so much for the response. Here I have provided some information.
a. This is clinical data which is around 859 in sample size.
b. It has 11 columns as input features and ...
- 11
0
votes
0
answers
18
views
XGBoost F1 score impovement methods for multi class classification [duplicate]
I am building a multiclass classification (5 classes) using XGBoost. Currently using 56 features for 1.6 million customer base having balanced classes.
The overall accuracy is 83%, F1 score is 0.81, ...
0
votes
0
answers
21
views
What is the maximum possible regression performance of a model on noisy data?
Lets say I am intersted in how fast something decays, so I measure its levels over time. I perform several replicates so I have several measurements at each time point, and then use that data to ...
- 311
0
votes
0
answers
22
views
Model comparison and experimentation for a thesis result
We are conducting a study to compare the accuracy of two computer vision models:
Model A: Trained on a non-augmented dataset of 11,200 real-world images.
Model B: Trained on an augmented dataset ...
- 101
3
votes
0
answers
21
views
Reward and Penalty Design in reinforcement learning
I hope you're all doing well.
I am currently working on a reinforcement learning problem to solve an optimization problem in wireless networks and I'm having troubles with designing the reward and ...
1
vote
0
answers
24
views
How best to approach ML classification active learning loop with ratio data?
I am currently on a project trying to investigate structure-property relationships between molecules and their phase behaviour (which is binary in this case). The molecules have a backbone, any number ...
- 11
0
votes
0
answers
28
views
Optimized algorithms for correlation based feature elimination
I have a large dataframe with close to a million rows and 2000 columns. I'm trying to do feature elimination using the correlation between the variables. The problem of course is that for a set of n ...
- 101
1
vote
0
answers
14
views
Preprocess two different kind of datasets for a machine learning problem
I am working on two health-related datasets. And I use Python.
One tabular dataset (called A) contains patient-level information (by id) and a bunch of other features which I have already transformed ...
- 55
4
votes
1
answer
68
views
How do machine learning topics fit into a traditional undergraduate statistics course on estimation?
I'm recently teaching an undergraduate introduction to statistics course, but as required by program director, need to add some machine learning materials to it. I'm wondering what is the appropriate ...
- 3,050
0
votes
0
answers
16
views
Youtube Spam Classifier - Different Methods yielding the same accuracy (94%)
(CONTEXT)
I'm currently doing a report project at my university to build a classifer model that classifies a comment as spam or ham (non-spam) using this data set, and then submit a prediction csv ...
- 389
0
votes
1
answer
21
views
Bayes Predictor for linear regression with square loss and expected value properties
I am trying to show that the Bayes predictor for linear regression with square loss is:
$$h^{\star}(x) = \mathbb{E}[Y|X = x]$$
I found the following slide from here, but don't understand which ...
- 101
0
votes
0
answers
30
views
Understanding heuristic-based outlier detection: concerns about scoring, weighting, and validity
I am trying to understand the mathematics and methodology behind a newly published outlier detection algorithm in the Computer & Security journal. This algorithm uses heuristic-based approaches, ...
- 445
1
vote
1
answer
39
views
Reasons and potential solutions for poor performance of elastic net penalized quantile regression
I'm performing elastic net penalized quantile regression (EN-QR) using the rqPen package on my dataset, which has 6,782 rows and 227 columns (i.e., predictors). 195 ...
2
votes
2
answers
37
views
Predicting the probability distribution of a deterministic dataset
In classical machine learning regression, we often assume the target variable $y$, given an input $x$, follows a probability distribution, allowing us to model and predict not just the expected value ...
- 121
3
votes
1
answer
29
views
Using statistical sampling and machine learning together?
I have data with labels $y_i \in \{0,1\}$ and some features $x_i$. Most $y$'s are 0 (e.g. 99% to 1%). I want to fit a random forest classification model on this data.
I was wondering if this approach ...
- 141
0
votes
1
answer
27
views
Separate Test Set for Cross-Validation for Small Sample (n=140)
I’m working on a survival analysis model with a small internal dataset (n=140). An outside researcher suggests splitting the dataset into train/val and setting aside a separate test set (e.g., ~10%, ...
- 1
6
votes
2
answers
486
views
Is my evaluation for this multiple linear regression correct?
I'm working on a multiple linear regression, following the tutorial by RegenerativeToday at https://www.youtube.com/watch?v=wH_ezgftiy0, using the insurance dataset at https://github.com/rashida048/...
- 163
5
votes
2
answers
102
views
Model intercomparison with no ground-truth labels
The domain is weather modeling. I've got 4 different models, one of which is mine, while the other 3 are independent models which I believe to be relatively skillful (i.e., much better than random). ...
- 101
3
votes
1
answer
166
views
Questions on backpropagation in a neural net
I understand how to symbolically apply back propagation, calculate the formulas with pen and paper. When it comes to actually using these derivations on data, I have 2 questions:
Suppose certain ...
- 599
0
votes
0
answers
31
views
Missing values in data set before DBSCAN
My goal is to identify bots and fraudulent users for an application. Ideally, this would be a regression problem where users are rated on a continuous scale. I have 4 tables that cover different ...
- 1
0
votes
0
answers
22
views
Forecasting Multiple steps of a Multivariate Time Series for ALL Features
I am working on a project where I have 100 multiple time series of length 1-10 minutes (samples every 0.1s). Each time series is a recording of human emotions stored as a vector of 7 features with ...
0
votes
0
answers
14
views
Why does my test loss and test evaluation metrics fluctuate?
I am fine-tuning the resnet18 model with additional classifiers. What I observed during the training process, is that test loss and other test evaluation metrics (AP, AUC) seem to fluctuate as you can ...
- 1
0
votes
1
answer
27
views
Choose a good estimator in a candidate set
Recently, I've been interested in the following statistical problem:
I have a set that consists of some estimators $\hat{A}_i$ of a matrix $A\in \mathbb{R}^{p\times p}$. Then I have some data ...
- 209
4
votes
1
answer
78
views
Analyses of Associations and Predictive Models in Random Forest
I am studying how Random Forests work to use this methodology in investigating the most important variables associated with a specific blood-based biomarker.
I would like some help to determine if my ...
0
votes
1
answer
28
views
GNNs with higher order adjacency matrices
Usually, the adjacency matrix stores information about direct connections of nodes in a graph.
The information from k-th neighbours is passed-on at k-th layers of GNNs, as described in the original ...
0
votes
0
answers
17
views
Why does the image classification model perform worse when augmenting only minority class
I have a problem of data imbalance (1:10 ratio) for image classification tasks.
To cope with it, I tried different imbalance training strategies, including weighted loss function, different loss ...
- 1
1
vote
1
answer
23
views
Multicollinearity (collinear predictors) checking is needed for non-linear regression (Poisson, negative binomial) and machine learning algorithms?
I am reading the book: Applied Predictive Modelling of Max Kuhn.
In part 3.5 Removing Predictors, the authors denoted that checking for Multicollinearity (collinear predictors) is important for linear ...
8
votes
4
answers
560
views
Signal-to-noise ratio in predictive modeling and machine learning
The interesting comments to this question get into how signal-to-noise ratio plays into ability to make predictions. Being more explicit about it, how does signal-to-noise ratio factor into how good ...
- 67.1k
0
votes
0
answers
25
views
Variable selection for checking casual relationship of regression model: should or should not? [duplicate]
I am looking for documents and online sources to understand whether or not I should exclude variables from my model through model selection (variable selection).
I also tried to use methods of Least ...
0
votes
1
answer
42
views
Should normalization be applied on interaction feature
I am working with interaction features in my machine learning model, where I create new features by multiplying a numeric variable with an encoded categorical feature. My question is:
Should ...
25
votes
2
answers
3k
views
Why doesn't ML suffer from curse of dimensionality?
Disclaimer: I asked this question on Data Science Stack Exchange 3 days ago, and got no response so far. Maybe it is not the right site. I am hoping for more positive engagement here.
This is a ...
- 1,825
1
vote
1
answer
44
views
Generalization error as U shape curve with respect to model complexity (bias variance tradeoff))
Is there any work mathematical rigorously prove that the generalization error for certain learning problems exhibits U shape curve with respect to model complexity (bias variance tradeoff)? Any ...
- 233
0
votes
0
answers
11
views
Identify predictors for clustering output?
I have a dataset with variables collected years ago, and many variables collected this year as outcome variables. I want to combine all the variables collected this year to get one outcome, e.g. ...
- 43
2
votes
1
answer
41
views
Alternatives for RMSE to Evaluate Goodness of Fit for Stable Distribution Parameters
I am estimating the parameters (alpha, beta, gamma, delta) of a stable distribution from a list of numerical data. I used a package to generate data from one type of stable distribution, specifically ...
- 121
0
votes
0
answers
37
views
Stacking for very high imbalanced class problem
Background: I'm facing a 1 : 40 000 class imbalanced problem.
It's a binary classification problem with positive class around ~500-700 instances and negative class in the tens of millions instances.
I ...
3
votes
1
answer
120
views
Custom Loss function Overfits to the Wrong Output but MSE Doesn't
I have a simple function that I want to approximate with a neural network:
N(1) = -1
N(2) = -1
N(3) = 1
N(4) = -1
Instead of using the MSE or cross-entropy losses, ...
- 33