Skip to main content

All Questions

Filter by
Sorted by
Tagged with
1 vote
0 answers
37 views

Number of runs needed for Probability of Improvement metric in Deep RL

I'm working with the Probability of Improvement (POI) metric described in [1], Section 4.3. The paper introduces various aggregate metrics in Section 4.3, and for most of these metrics (IQM, mean, ...
desert_ranger's user avatar
0 votes
0 answers
8 views

Predicting FPL Player Total Points using Random Forest

I have a dataset with around 100k of gameweek stats in the English Premier League (from 2016-2023). My goal is to predict how many total points a player will score in a certain gameweek/match. I ...
user avatar
0 votes
1 answer
31 views

Fitting a Nonlinear Mixed Model

I’m trying to fit a nonLinear Mixed Model (nLMM) to test whether the abundance of certain organisms was affected by the sampling period after an event that caused a significant increase. The data show ...
Pablo's user avatar
  • 33
5 votes
1 answer
115 views

XGBoost/ XGBRanker to produce probabilities instead of ranking scores

I have a dataset of the performance of students in exams which looks like: ...
Ishigami's user avatar
  • 185
1 vote
1 answer
21 views

Two questions about the VC theory (on the generalization error bound)

In Andrews Ng's machine learning notes (https://cs229.stanford.edu/main_notes.pdf), he introduced the following bound for the difference between generalization error and training error (see the ...
ExcitedSnail's user avatar
  • 3,050
1 vote
0 answers
52 views

Why do machine learning courses on regression mostly focus on gradient descient although we have the closed form estimator $(X'X)^{-1}X'Y$? [duplicate]

In many online machine learning courses and videos(such as Andrew Ng's coursera course), when it comes to regression (for example regressing $Y$ on features $X$), althouth we have the closed form ...
ExcitedSnail's user avatar
  • 3,050
1 vote
0 answers
32 views

Preprocessing and model selection strategies

I am working on a fault detection problem where each sample is a time series labeled with a specific type of fault. I am using a CNN model and a validation set for hyperparameter tuning. Currently, I ...
S.H.W's user avatar
  • 77
5 votes
1 answer
113 views
+50

How to find a de-biased estimator with a ML component in my contaminated data problem?

I am trying to use the output of a machine learning model to estimate (using a maximum likelihood approach) a parameter in a distribution. The estimator I get has a bias which is much larger than the ...
Ori's user avatar
  • 101
6 votes
2 answers
684 views

Building a Statistically Sound ML Model

Silent reader here in the statistics substack. One thing I've learned is that many "default" machine learning practices are being challenged due to fundamental statistical mistakes. This has ...
easymoneysniper's user avatar
2 votes
1 answer
37 views

Handling a very informative feature with significant missing values

I have a machine learning model where the goal is prediction in the context of regression. For my metric of interest, there is a feature which is extremely informative but has significant missing ...
Adam's user avatar
  • 498
1 vote
2 answers
39 views

How does a single layer/single unit with Adam optimizer network work?

I'm very new to ML and I'm trying to mess around with Linear Regression. I tested sklearn's LinearRegression model and then wanted to compare the results to a very simple neural network. I created a ...
TamerM's user avatar
  • 111
6 votes
1 answer
170 views

Why the loss is not considered as a "supervisory signal" in unsupervised learning?

It is said that supervised is different from unsupervised learning due to the presence of "supervisory signals" aka labels. However, in both cases we have a loss function. Isn't the loss a ...
Antonios Sarikas's user avatar
0 votes
0 answers
15 views

Comparing probabilities of two models

Consider a dataset and two binary classes CLASS_A and CLASS_B, with different proportions of 0 and 1. Suppose we train a model such as XGBClassifier for both ...
Ale's user avatar
  • 1,690
0 votes
0 answers
8 views

Statistical Testing with Minimal Samples for Reinforcement Learning Algorithms

I'm working on comparing two reinforcement learning algorithms where: Running experiments is extremely computationally expensive Based on preliminary results, Algorithm B consistently and ...
desert_ranger's user avatar
-1 votes
1 answer
76 views

Categorical Dependent Variable

Repost: Hello all, thank you so much for the response. Here I have provided some information. a. This is clinical data which is around 859 in sample size. b. It has 11 columns as input features and ...
Ayesha Haya's user avatar
0 votes
0 answers
18 views

XGBoost F1 score impovement methods for multi class classification [duplicate]

I am building a multiclass classification (5 classes) using XGBoost. Currently using 56 features for 1.6 million customer base having balanced classes. The overall accuracy is 83%, F1 score is 0.81, ...
saurabh15's user avatar
0 votes
0 answers
21 views

What is the maximum possible regression performance of a model on noisy data?

Lets say I am intersted in how fast something decays, so I measure its levels over time. I perform several replicates so I have several measurements at each time point, and then use that data to ...
Ian Sudbery's user avatar
0 votes
0 answers
22 views

Model comparison and experimentation for a thesis result

We are conducting a study to compare the accuracy of two computer vision models: Model A: Trained on a non-augmented dataset of 11,200 real-world images. Model B: Trained on an augmented dataset ...
markcalendario's user avatar
3 votes
0 answers
21 views

Reward and Penalty Design in reinforcement learning

I hope you're all doing well. I am currently working on a reinforcement learning problem to solve an optimization problem in wireless networks and I'm having troubles with designing the reward and ...
Mehran Varshosaz's user avatar
1 vote
0 answers
24 views

How best to approach ML classification active learning loop with ratio data?

I am currently on a project trying to investigate structure-property relationships between molecules and their phase behaviour (which is binary in this case). The molecules have a backbone, any number ...
user6277's user avatar
0 votes
0 answers
28 views

Optimized algorithms for correlation based feature elimination

I have a large dataframe with close to a million rows and 2000 columns. I'm trying to do feature elimination using the correlation between the variables. The problem of course is that for a set of n ...
SacredMechanic's user avatar
1 vote
0 answers
14 views

Preprocess two different kind of datasets for a machine learning problem

I am working on two health-related datasets. And I use Python. One tabular dataset (called A) contains patient-level information (by id) and a bunch of other features which I have already transformed ...
hiu's user avatar
  • 55
4 votes
1 answer
68 views

How do machine learning topics fit into a traditional undergraduate statistics course on estimation?

I'm recently teaching an undergraduate introduction to statistics course, but as required by program director, need to add some machine learning materials to it. I'm wondering what is the appropriate ...
ExcitedSnail's user avatar
  • 3,050
0 votes
0 answers
16 views

Youtube Spam Classifier - Different Methods yielding the same accuracy (94%)

(CONTEXT) I'm currently doing a report project at my university to build a classifer model that classifies a comment as spam or ham (non-spam) using this data set, and then submit a prediction csv ...
KitanaKatana's user avatar
0 votes
1 answer
21 views

Bayes Predictor for linear regression with square loss and expected value properties

I am trying to show that the Bayes predictor for linear regression with square loss is: $$h^{\star}(x) = \mathbb{E}[Y|X = x]$$ I found the following slide from here, but don't understand which ...
Victor Hugo's user avatar
0 votes
0 answers
30 views

Understanding heuristic-based outlier detection: concerns about scoring, weighting, and validity

I am trying to understand the mathematics and methodology behind a newly published outlier detection algorithm in the Computer & Security journal. This algorithm uses heuristic-based approaches, ...
Mario's user avatar
  • 445
1 vote
1 answer
39 views

Reasons and potential solutions for poor performance of elastic net penalized quantile regression

I'm performing elastic net penalized quantile regression (EN-QR) using the rqPen package on my dataset, which has 6,782 rows and 227 columns (i.e., predictors). 195 ...
Kevin Sanchez's user avatar
2 votes
2 answers
37 views

Predicting the probability distribution of a deterministic dataset

In classical machine learning regression, we often assume the target variable $y$, given an input $x$, follows a probability distribution, allowing us to model and predict not just the expected value ...
juekai's user avatar
  • 121
3 votes
1 answer
29 views

Using statistical sampling and machine learning together?

I have data with labels $y_i \in \{0,1\}$ and some features $x_i$. Most $y$'s are 0 (e.g. 99% to 1%). I want to fit a random forest classification model on this data. I was wondering if this approach ...
urnproblems's user avatar
0 votes
1 answer
27 views

Separate Test Set for Cross-Validation for Small Sample (n=140)

I’m working on a survival analysis model with a small internal dataset (n=140). An outside researcher suggests splitting the dataset into train/val and setting aside a separate test set (e.g., ~10%, ...
mel's user avatar
  • 1
6 votes
2 answers
486 views

Is my evaluation for this multiple linear regression correct?

I'm working on a multiple linear regression, following the tutorial by RegenerativeToday at https://www.youtube.com/watch?v=wH_ezgftiy0, using the insurance dataset at https://github.com/rashida048/...
SVill's user avatar
  • 163
5 votes
2 answers
102 views

Model intercomparison with no ground-truth labels

The domain is weather modeling. I've got 4 different models, one of which is mine, while the other 3 are independent models which I believe to be relatively skillful (i.e., much better than random). ...
jr15's user avatar
  • 101
3 votes
1 answer
166 views

Questions on backpropagation in a neural net

I understand how to symbolically apply back propagation, calculate the formulas with pen and paper. When it comes to actually using these derivations on data, I have 2 questions: Suppose certain ...
Baron Yugovich's user avatar
0 votes
0 answers
31 views

Missing values in data set before DBSCAN

My goal is to identify bots and fraudulent users for an application. Ideally, this would be a regression problem where users are rated on a continuous scale. I have 4 tables that cover different ...
Burger's user avatar
  • 1
0 votes
0 answers
22 views

Forecasting Multiple steps of a Multivariate Time Series for ALL Features

I am working on a project where I have 100 multiple time series of length 1-10 minutes (samples every 0.1s). Each time series is a recording of human emotions stored as a vector of 7 features with ...
Arnav Tapadia's user avatar
0 votes
0 answers
14 views

Why does my test loss and test evaluation metrics fluctuate?

I am fine-tuning the resnet18 model with additional classifiers. What I observed during the training process, is that test loss and other test evaluation metrics (AP, AUC) seem to fluctuate as you can ...
Yuju Ahn's user avatar
0 votes
1 answer
27 views

Choose a good estimator in a candidate set

Recently, I've been interested in the following statistical problem: I have a set that consists of some estimators $\hat{A}_i$ of a matrix $A\in \mathbb{R}^{p\times p}$. Then I have some data ...
mathhahaha's user avatar
4 votes
1 answer
78 views

Analyses of Associations and Predictive Models in Random Forest

I am studying how Random Forests work to use this methodology in investigating the most important variables associated with a specific blood-based biomarker. I would like some help to determine if my ...
always.learning's user avatar
0 votes
1 answer
28 views

GNNs with higher order adjacency matrices

Usually, the adjacency matrix stores information about direct connections of nodes in a graph. The information from k-th neighbours is passed-on at k-th layers of GNNs, as described in the original ...
ignoramus's user avatar
0 votes
0 answers
17 views

Why does the image classification model perform worse when augmenting only minority class

I have a problem of data imbalance (1:10 ratio) for image classification tasks. To cope with it, I tried different imbalance training strategies, including weighted loss function, different loss ...
Yuju Ahn's user avatar
1 vote
1 answer
23 views

Multicollinearity (collinear predictors) checking is needed for non-linear regression (Poisson, negative binomial) and machine learning algorithms?

I am reading the book: Applied Predictive Modelling of Max Kuhn. In part 3.5 Removing Predictors, the authors denoted that checking for Multicollinearity (collinear predictors) is important for linear ...
Student coding's user avatar
8 votes
4 answers
560 views

Signal-to-noise ratio in predictive modeling and machine learning

The interesting comments to this question get into how signal-to-noise ratio plays into ability to make predictions. Being more explicit about it, how does signal-to-noise ratio factor into how good ...
Dave's user avatar
  • 67.1k
0 votes
0 answers
25 views

Variable selection for checking casual relationship of regression model: should or should not? [duplicate]

I am looking for documents and online sources to understand whether or not I should exclude variables from my model through model selection (variable selection). I also tried to use methods of Least ...
Student coding's user avatar
0 votes
1 answer
42 views

Should normalization be applied on interaction feature

I am working with interaction features in my machine learning model, where I create new features by multiplying a numeric variable with an encoded categorical feature. My question is: Should ...
Harshit Goyal's user avatar
25 votes
2 answers
3k views

Why doesn't ML suffer from curse of dimensionality?

Disclaimer: I asked this question on Data Science Stack Exchange 3 days ago, and got no response so far. Maybe it is not the right site. I am hoping for more positive engagement here. This is a ...
Landon Carter's user avatar
1 vote
1 answer
44 views

Generalization error as U shape curve with respect to model complexity (bias variance tradeoff))

Is there any work mathematical rigorously prove that the generalization error for certain learning problems exhibits U shape curve with respect to model complexity (bias variance tradeoff)? Any ...
Hao Yu's user avatar
  • 233
0 votes
0 answers
11 views

Identify predictors for clustering output?

I have a dataset with variables collected years ago, and many variables collected this year as outcome variables. I want to combine all the variables collected this year to get one outcome, e.g. ...
NPpsy's user avatar
  • 43
2 votes
1 answer
41 views

Alternatives for RMSE to Evaluate Goodness of Fit for Stable Distribution Parameters

I am estimating the parameters (alpha, beta, gamma, delta) of a stable distribution from a list of numerical data. I used a package to generate data from one type of stable distribution, specifically ...
Danny Wen's user avatar
  • 121
0 votes
0 answers
37 views

Stacking for very high imbalanced class problem

Background: I'm facing a 1 : 40 000 class imbalanced problem. It's a binary classification problem with positive class around ~500-700 instances and negative class in the tens of millions instances. I ...
user24758287's user avatar
3 votes
1 answer
120 views

Custom Loss function Overfits to the Wrong Output but MSE Doesn't

I have a simple function that I want to approximate with a neural network: N(1) = -1 N(2) = -1 N(3) = 1 N(4) = -1 Instead of using the MSE or cross-entropy losses, ...
Andrew Baker's user avatar

1
2 3 4 5
419