Skip to main content

New answers tagged

0 votes

Why can't this function be used as a loss function?

While error rate can't be used for gradient descent, there are so called 0-1 relaxations, that approximate error rate during training. For the binary case look at the following work: Nguyen, Tan, and ...
Ivan Karpukhin's user avatar
0 votes

Is more data really always better in machine learning?

A short answer - No. The question "is more data provably improve the fit" is a great question, and it can be interpreted in many ways. One natural way is to ask: Is adding the expected loss ...
Amit Keinan's user avatar
0 votes

Customized distribution fitting

I stand by what I wrote here about the Princess and the Pea. You pretty much know that whatever distribution you fit will be at least a little bit incorrect. Then you ask the hypothesis test if that ...
Dave's user avatar
  • 65.1k
0 votes

Fair comparison method for a biased physics-based model and its ML-correction version

Since the comments are getting crowded, I'm creating an answer instead, but these are mostly small remarks. What's the goal of the comparison? it's unclear to me why we are comparing the methods, ...
Guillaume Dehaene's user avatar
1 vote

I have a dataset with 18 biomarker features and a target variable. I want to find the features which are having the biggest impact on the target

First, avoid feature selection. That's particularly true for binary regression models, as omitting any outcome-associated predictor from the model leads to bias in estimates for the included ...
EdM's user avatar
  • 95.8k
0 votes
Accepted

Threshold selection in chain classifier

Can be solved by solving a mixed integer programming problem described on this Math Stackexchange answer.
Bait Hoven's user avatar
0 votes

Does It Make Sense to Use Random Forest When Predictor Values Are Averaged by Groups?

There's a few problems here and I would not recommend random forests for this. Effectively, you have many technical replicates of diversity measurements at 5 locations. This is certainly better than ...
mkt's user avatar
  • 18.9k
0 votes

Should I Use Regularization in Univariate Logistic Regression for Diagnostic Methods Comparison?

I assume your goal is to predict Method 2 with Method 1's value. Regularization shrinks the parameter to 0. As you have only one input, the shrinkage is less meaningful. I view the regularization ...
Xiaochuan Lu's user avatar
2 votes
Accepted

Link between Cross-entropy and MLE

As you correctly claimed both MLE and CE give the exactly same optimal model parameters (at least for iid cases), there's no theoretical advantage of either objective to learn the usual point estimate ...
cinch's user avatar
  • 436
1 vote

How to generate 95% prediction interval around predictions from ML model?

Simple linear regression is parametric, since you model $Y = \beta X + \epsilon$ with the assumption of $\epsilon \sim N(0, \sigma^2)$. The prediction interval for a new $\hat y_h$ is bigger than the ...
qwr's user avatar
  • 548
0 votes

Defining clinical follow-up: Fixed Period vs. Maximum Duration

It depends on what you are trying to prove. If you are trying to test that complication rates have decreased over the years (or at a minimnum test if they have changed over time), then you need to ...
jginestet's user avatar
  • 1,934
0 votes

Can I apply data augmentation to the test set?

Such or similar techniques have been described in the literature, e.g.: U. Braga-Neto and E. Dougherty: Bolstered error estimation, Pattern Recognition 2004, DOI 10.1016/j.patcog.2003.08.017 uses a (...
cbeleites's user avatar
  • 39.3k
3 votes
Accepted

How to generate 95% prediction interval around predictions from ML model?

THIS SEEMS TO BE AN OPEN PROBLEM. Let's look at some possible solutions and their drawbacks. First, you propose this Yhat +- 1.96 * std(residuals). Let's put that ...
Dave's user avatar
  • 65.1k
2 votes
Accepted

Defining clinical follow-up: Fixed Period vs. Maximum Duration

You are effectively performing "survival analysis," but your event is complication rather than death. Tools for survival analysis can be used for any defined type of event if you want to ...
EdM's user avatar
  • 95.8k
1 vote

Multi Level / Hierarchical Time Series Models in Python

STAN may be an option. It has a Python interface. see: https://mc-stan.org/docs/stan-users-guide/regression.html#hierarchical-regression
user1747134's user avatar
0 votes

Can I apply data augmentation to the test set?

Frans Rodenburg already gave a great answer (I especially second the K-fold cross-validation). I just want to add two points to it. First, if you're going to add Gaussian noise to your test set, all ...
Ruben van Bergen's user avatar
3 votes

Can I apply data augmentation to the test set?

Data splitting is not stable, especially not with such a small sample size. Data augmentation is typically used to show your model multiple variations of the original observations, all slightly ...
Frans Rodenburg's user avatar
3 votes

What is the Gold Standard for Evaluating the Posterior of a Bayesian Regression Model?

There are multiple senses of "correct" you may want to invoke. A) - good predictions. If by "correct" you mean "has useful predictions", than cross validation (or better ...
Martin Modrák's user avatar
0 votes

Overfitting the (non-nested) cross validation set

We may do model selection OR hyper-parameter tuning using non-nested cross validation. Model selection vs Hyper Parameter tuning. Suppose we are doing model selection. Then we train on k-1 folds and ...
user2338823's user avatar
7 votes
Accepted

What is the Gold Standard for Evaluating the Posterior of a Bayesian Regression Model?

The method of choice to evaluate probabilistic predictions is a proper scoring rule. The Brier score is one such, for the special case of 0-1 outcomes - that is, classification. However, there are ...
Stephan Kolassa's user avatar
0 votes

How can different models based on different sets of predictors be combined to significantly improve the model performance?

Hi there: What was the logic behind the combination of the two sets of predictor variables? The answer to your question lies there. That logic must guide your selection of which predictors to combine, ...
Bill Luker Jr's user avatar
1 vote

Mathematical Prediction of Linear Mixed Models Random Intercept

Below I will explain the idea behind a method used to predict the random intercept in a multilevel model. So my answer is about this underlying idea, and not about a fast way to calculate the ...
BenP's user avatar
  • 1,838
1 vote
Accepted

Weight initialisation for neural networks - should they be different for each observations or the same?

Thanks to @Dave's comment, I now know that I should initialise the weights only once for all samples (indeed, it does not make much sense to do otherwise). By the way, just a curiosity: I was confused ...
umbe1987's user avatar
  • 297
0 votes

ADALINE simple implementation with 2 features bug

Two mistakes: ADALINE needs the outputs to be encoded as $\pm 1$, not as $\{0, 1\}$, if you want your class boundary equation to be the way you stated it; and, consequently the threshold for ...
Igor F.'s user avatar
  • 9,418
1 vote

How can I know how 'descriptive' my data is?

In a way, you are asking about something similar to effect sizes, which is defined as: $$ d = \frac{(\mu_1 - \mu_2)}{\sigma} $$ where $\mu_1$ is the mean of treatment 1, $\mu_2$ is the mean of ...
Ggjj11's user avatar
  • 1,543
3 votes

Fitting multiple linear regression models to select molecules for which a feature of interest significantly alters concentration

A newly edited version of this question clarifies that the interest is in identifying which among 92 proteins have differential expression between 16 normal controls and 16 patients, with the patients ...
EdM's user avatar
  • 95.8k
1 vote
Accepted

Predictive Maintenance of factory parts

Given that you know the periodicity, I would suggest that you aim to make a model to predict the 24-hour sliding average, and another to predict the daily variation. For the latter, the training set ...
chrishmorris's user avatar
  • 1,855
0 votes

The meaning of linear transformation in a batch norm revisited

Introduction I think your reasoning regarding the interpretation of the linear transformation in batch normalisation (BatchNorm) is generally correct. Let's break it down to clarify the concepts: ...
Robert Long's user avatar
  • 64.1k
2 votes
Accepted

Modeling a functional relationship with Constrained Gaussian Process regression

To address the problem of modeling a nonlinear relationship with a response variable $y \in (0,1)$, one viable approach is to use a Gaussian Process (GP) model while transforming the response variable ...
Robert Long's user avatar
  • 64.1k
2 votes

Fitting multiple linear regression models to select molecules for which a feature of interest significantly alters concentration

If I understood you research question, it seems like you’re navigating a problem involving identifying molecules that are altered between two categorical classes while controlling for a continuous ...
Robert Long's user avatar
  • 64.1k
2 votes

What does "Aleatoric and Epistemic uncertainties" mean?

Aleatoric Uncertainty: This is the uncertainty of the process which you are trying to model. Say, you want to train a model with some sensor output where the sensor is itself producing some random ...
hafiz031's user avatar
  • 235
5 votes
Accepted

Mathematical Prediction of Linear Mixed Models Random Intercept

For a random intercepts model, the formula for the empirical Bayes estimate of the random effect is $$\hat{u}_i = \frac{\hat{\sigma}_u^2}{\hat{\sigma}_\epsilon^2 + n_i \hat{\sigma}_u^2} \sum\limits_{j ...
Dimitris Rizopoulos's user avatar
4 votes

Is duplicating dataset an augmentation?

Duplicating the dataset without changing anything is a bad idea. It does nothing useful (no augmentation is done, no new information added), but pollutes the out-of-bag (i.e. when you randomly sample ...
Björn's user avatar
  • 33.6k
1 vote

Is duplicating dataset an augmentation?

I don't think it's reasonable to duplicate existing data and call it data augmentation. Augmentation as I understand it involves some sort of transformation to the data (rotating images, adding noise, ...
mkt's user avatar
  • 18.9k
1 vote

tree plotted by ctree has no splits

If no splits are implemented, this generally indicates there is no association between the response (the PC in this case) and possible predictors (environmental variables in this case). You might want ...
Marjolein Fokkema's user avatar
0 votes

Neural networks output probability estimates?

I am not a data scientist, so my answer may not be very useful; however, I am facing the same question, and my idea is to actually calculate the probability on the validation subset of data once the ...
fede72bari's user avatar
0 votes

how to train and hypertune a model

Setting the random state is a tool for Preventing you from chasing small differences in the validation outcomes Making sure other researchers can reproduce your result exactly Second one is more ...
Gijs's user avatar
  • 3,889
5 votes
Accepted

How can I learn and remove the linear trend in the residuals against the true response values generated by an ordinary neural network?

You should have linear correlation between $y$ and $y-\hat y$. What you don't want is linear correlation between $\hat y$ and $y-\hat y$. Suppose you could get the best possible prediction from a set ...
Thomas Lumley's user avatar
0 votes

Levenshtein/Edit Distance as a loss function for sequence transformer models?

You might be interested in the CTC loss. See also https://ogunlao.github.io/blog/2020/07/17/breaking-down-ctc-loss.html.
D.W.'s user avatar
  • 6,688

Top 50 recent answers are included